Disk-drive enclosure having drives in a herringbone pattern to improve airflow and method

ABSTRACT

A system and method for an improved multiple hard-disk-drive data-storage enclosure. Some embodiments position drives in counter-rotating pairs, each simultaneously accessing half the data, such that seek-caused actuator rotational-acceleration vibration cause simultaneous canceling rotational torque. Some embodiments position the edge of a first drive (or drive pair) at an angle to the actuator midpoint of a nearby second drive (or drive pair), such that rotational-acceleration vibration from a seek-caused actuator rotation in the first drive does not cause a rotational movement into the second drive that affects the tracking or seek operation. Some further embodiments position drives in a herringbone pattern to redirect air flow in addition to reducing rotational-acceleration vibration interaction. Other embodiments include a printed wire circuit board mounted to reduce the rotational-acceleration vibration interaction.

CROSS-REFERENCES TO RELATED INVENTIONS

This is a divisional of U.S. patent application Ser. No. 11/027,777,filed Dec. 29, 2004 now U.S. Pat. No. 7,167,359 and titled “SYSTEM ANDMETHOD FOR MASS STORAGE USING MULTIPLE-HARD-DISK-DRIVE ENCLOSURE,” whichclaims benefit of U.S. Provisional Patent Application No. 60/580,987,filed Jun. 18, 2004 and titled “SYSTEM AND METHOD FOR REDUCED VIBRATIONINTERACTION IN A MULTIPLE-HARD-DISK-DRIVE ENCLOSURE,” and of U.S.Provisional Patent Application No. 60/533,605, filed Dec. 29, 2003 andtitled “SYSTEM AND METHOD FOR IMPROVED HARD-DISK-DRIVE DATA-STORAGEENCLOSURE,” each of which is hereby incorporated by reference in itsentirety. This application is also related to U.S. patent applicationSer. No. 11/026,553, filed Dec. 29, 2004 and titled “SYSTEM AND METHODFOR REDUCED VIBRATION INTERACTION IN A MULTIPLE-DISK-DRIVE ENCLOSURE,”which is incorporated herein by reference in its entirety.

This application is additionally related to:

-   U.S. patent application Ser. No. 11/537,600 filed on Sep. 29, 2006    and entitled “DISK-DRIVE ENCLOSURE HAVING FRONT-BACK ROWS OF    SUBSTANTIALLY PARALLEL DRIVES AND METHOD”;-   U.S. patent application Ser. No. 11/537,605 filed on Sep. 29, 2006    and entitled “DISK-DRIVE ENCLOSURE HAVING ROWS OF ALTERNATELY FACING    PARALLEL DRIVES TO REDUCE VIBRATION AND METHOD”;-   U.S. patent application Ser. No. 11/537,606 filed on Sep. 29, 2006    and entitled “DISK-DRIVE ENCLOSURE HAVING LATERALLY OFFSET PARALLEL    DRIVES TO REDUCE VIBRATION AND METHOD”;-   U.S. patent application Ser. No. 11/537,608 filed on Sep. 30, 2006    and entitled “DISK-DRIVE ENCLOSURE HAVING NON-PARALLEL DRIVES TO    REDUCE VIBRATION AND METHOD”;-   U.S. patent application Ser. No. 11/537,614 filed on Sep. 30, 2006    and entitled “DISK-DRIVE ENCLOSURE HAVING PAIR-WISE COUNTER-ROTATING    DRIVES TO REDUCE VIBRATION AND METHOD”;-   U.S. patent application Ser. No. 11/537,603 filed on Sep. 29, 2006    and entitled “DISK-DRIVE SYSTEM HAVING MULTIPLE POWER SUPPLIES AND    MIRRORING AND METHOD”;-   U.S. patent application Ser. No. 11/537,607 filed on Sep. 30, 2006    and entitled “DISK-DRIVE SYSTEM SUPPORTING MASSIVELY PARALLEL VIDEO    STREAMS AND METHOD”;-   U.S. patent application Ser. No. 11/537,610 filed on Sep. 30, 2006    and entitled “DISK-DRIVE SYSTEMS WITH A VARYING NUMBER OF SPARES FOR    DIFFERENT EXPECTED LIFETIMES AND METHOD”; and-   U.S. patent application Ser. No. 11/537,613 filed on Sep. 30, 2006    and entitled “POROUS LIGHT-EMITTING DISPLAY WITH AIR FLOW THROUGH    DISPLAY, ITS USE IN A DISK-DRIVE SYSTEM AND METHOD”; which are all    hereby incorporated by reference in their entirety.

FIELD OF THE INVENTION

This invention relates to hard-disk-drive data-storage systems andmethods, and more particularly, to enclosures that hold a large numberof hard-disk-drives and provide a large number of serial data interfacesoperating in parallel, resulting in, among other things, improvedperformance, reliability, manufacturing costs, and/or operational costs.

BACKGROUND OF THE INVENTION

Massive amounts of data storage are required for many emerging andexisting applications. For example, video-on-demand applications canprovide access to hundreds or thousands of movies for hundreds orthousands of users simultaneously, requiring vast amounts of digitalstorage, fast access, 24 hours-per-day and 7 days per week (24/7)availability and uptime, and huge bandwidth. Modern supercomputers alsorequire these features, as well as requiring even faster access,extraordinary data integrity, error checking and error correction.

Semiconductor memories provide very fast access, reasonable densities,and moderate costs. However, most common semiconductor memories arevolatile (they lose their data when not powered or not refreshed on atimely basis), they develop soft errors (errors that can be corrected byre-writing the affected location) due to various causes including alpharadiation, and they can be cost prohibitive. Additionally, the heat andpower requirements can be problematic, if they are used to store vastamounts of information for long time periods.

Hard-disk drives (HDDs, also called just “disk drive” or “drive”))provide cost-effective non-volatile data storage on rotating media. Dataare writ en and read by magnetic transducer heads that are moved to oneof thousands of tracks to locate requested data. There are timepenalties incurred to move the head to the requested track, to rotatethe disk to the requested location on that track, and to serially reador write the data from or to the track location. The moving parts of adisk drive are prone to wear and failure over time. For applicationsrequiring high reliability (error-free data) and availability (24/7uptime), data can be stored in a redundant manner (e.g., redundantarrays of inexpensive disks, or RAID), and several different RAIDschemes are known to the art, frequently making compromises betweenperformance, cost, and data recoverability. Another requirement for manyapplications is serviceability—the ease of repairing a faulty system inthe field (i.e., at a customer's location of the equipment).

Data storage servers (enclosures having one or more disk drives as wellas a data processor to receive data access requests and control thestoring and fetching of data to and from the disk drives) and storagevaults (enclosures having one or more disk drives but essentially noprocessor, and using a data processor housed in a separate enclosure toreceive data access requests and control the storing and fetching ofdata to and from the disk drives) can be implemented in free-standingunits (typically an upright unit placed on the floor or on a desk) or asrack-mount units (typically horizontally-oriented units bolted to astandardized nineteen-inch (48.26 cm) rack).

Typical conventional rack-mount disk-drive enclosures arrange aplurality (3 to 14) HDDs in removable carriers that are accessible fromthe “front” of the unit (the side typically facing a user area), andusually are arranged so that data and power cables are accessible fromthe “back” of the unit. The disk drives can thus be replaced fairlyeasily if one were to fail. RAID solutions can be utilized to useredundant data artifacts to compute the data that was on the failed diskdrive. This data is sent to a requester or used to recreate the data ona new (spare) disk drive once one is inserted to replace the failedunit. Since racks of rack-mount units are often installed in rows, thereis typically no access provided from the sides of a rack-mount unit, andsince the rack-mount units are stacked one on top of another in eachrack there is typically no access provided from the top or bottom of arack-mount unit.

High-density packaging of HDDs in an enclosure exacerbatesdrive-to-drive vibration interaction problems. With several HDDs,packaged closely together in single enclosure, potentially many doingsimultaneous head-seeks, the vibration interaction problem is greatlyincreased. Previous systems and methods to package HDDs and reducedrive-to-drive vibration interaction involved mechanical stiffening ofthe enclosure and/or lower density packaging options.

Numerous computer applications utilize multiple disk drives for datastorage and acquisition. These multiple disk drives are often located inseparated locations. For example, disk drives may be arranged in racksystems that consume large amounts of space and require multiplecabinets to house the rack systems. Furthermore, positioning multipledisk drives in separate locations adds to the complexity of dataacquisition from the disk drives because a more complex interface withthe multiple disk drives is required. In addition, longer cabling isrequired to reach the separately located disk drives. Accordingly, whatis needed is an apparatus that positions multiple disk drives in amanner that simplifies data acquisition from the disk drives and reducesthe space needed to house the multiple disk drives.

SUMMARY OF THE INVENTION

In some embodiments, the present invention generally involves housing alarge number of disk drives in an enclosure. In other embodiments, theinvention is based on positioning disk drives such that forces occurringduring seek and write functions within a first disk drive arecounteracted by analogous forces occurring in one or more other drivesthat are positionally paired with the first disk drive in someembodiments. An example of such a force includes rotation andcounter-rotation of disks that is caused by movement of an actuator armwithin the disk drive that occurs during a seek or write function of thedisk. Other examples of such forces include vibrational forces,rotational, counter-rotational forces, and the like that are due to themovement of a disk within a disk drive. These forces can be caused bynumerous actions within a disk drive. Arranging the disk drivesaccording to the invention helps to reduce detrimental results caused bysuch forces that can increase the incidence of read and write errors.Accordingly, the invention can be used to position multiple disk drivesso that the disk drives have a reduced read and write error rate.

In some embodiments, the invention provides an apparatus that includes asubstrate, and a plurality of disk drives each coupled electrically andmechanically to the substrate, the plurality of disk drives including atleast a first and a second disk drive, wherein the first disk drive ispositioned relative to the second disk drive so that a rotational forceproduced by the first disk drive is at least partially counteracted by arotational force produced by the second disk drive.

In other embodiments, the invention provides a method that includesmounting a plurality of drives in an enclosure, the enclosure includinga connector substrate, the plurality of drives including at least afirst disk drive and a second disk drive that are each electrically andmechanically coupled to the enclosure; and mechanically coupling thefirst drive and the second drive such that rotational force produced bythe first disk drive is at least partially counteracted by rotationalforce produced by the second disk drive.

In some embodiments, the invention provides an apparatus that includesan enclosure that includes a substrate, a means in the enclosure formounting a plurality of disk drives to the enclosure, and a means forcoupling a plurality of disk drives electrically and mechanically to thesubstrate, the plurality of disk drives including at least a first and asecond disk drive, wherein the first disk drive is positioned relativeto the second disk drive so that a rotational force produced by thefirst disk drive is at least partially counteracted by a rotationalforce produced by the second disk drive.

In some embodiments, the invention provides an apparatus that includes asubstrate, and a plurality of disk drives each coupled electrically andmechanically to the substrate, the plurality of disk drives including atleast a first disk drive and a second disk drive, wherein the first andsecond disk drive each have a first major face surrounded by a first,second, third and fourth edge and having a first, second, third andfourth corner, wherein the first disk drive and the second disk driveare positioned such that a rotational force produced by the first diskdrive is conveyed primarily as a translational force to the second diskdrive.

In some embodiments, the invention provides a method that includesmounting a plurality of drives in an enclosure, the plurality of drivesincluding at least a first disk drive and a second disk drive that areeach electrically and mechanically coupled to the enclosure, andmechanically coupling the first disk drive and the second disk drivesuch that rotational force produced by the first disk drive is at leastpartially transmitted as translational force to the second disk drive.

In some embodiments, the invention provides an apparatus that includes asubstrate; and a means for mounting a plurality of disk drives to thesubstrate; and a means for coupling a plurality of disk driveselectrically and mechanically to the substrate, the plurality of diskdrives including at least a first disk drive and a second disk drive,wherein the first and second disk drive each have a first major facesurrounded by a first, second, third and fourth edge and having a first,second, third and fourth corner, wherein the first disk drive and thesecond disk drive are positioned such that a rotational force producedby the first disk drive is conveyed primarily as a translational forceto the second disk drive.

In some embodiments, the invention provides an apparatus that includes asubstrate, and a plurality of disk-drive connectors each coupledelectrically and mechanically to the substrate, the plurality ofdisk-drive connectors including at least a first and a second disk-driveconnector, wherein the first disk-drive connector is positioned relativeto the second disk-drive connector so that a rotational force producedby a first disk drive that is connected to the first disk-driveconnector is at least partially counteracted by a rotational forceproduced by a second disk drive that is connected to the seconddisk-drive connector.

In some embodiments, the invention provides an apparatus that includes asubstrate, and a plurality of disk-drive connectors each coupledelectrically and mechanically to the substrate, the plurality ofdisk-drive connectors including at least a first disk-drive connectorand a second disk-drive connector, wherein the first disk-driveconnector and the second disk-drive connector are positioned such that arotational force produced by a first disk drive connected to the firstdisk-drive connector is conveyed primarily as a translational force to asecond disk drive connected to the second disk-drive connector.

In some embodiments, the invention provides a method that includesmounting a plurality of disk-drive connectors in an enclosure, theenclosure including a connector substrate, the plurality of disk-driveconnectors including at least a first disk-drive connector and a seconddisk-drive connector that are each electrically and mechanically coupledto the enclosure, and mechanically coupling the first disk-driveconnector and the second disk-drive connector such that rotational forceproduced by a first disk drive that is connected to the first disk-driveconnector is at least partially counteracted by rotational forceproduced by a second disk drive that is connected to the seconddisk-drive connector.

In some embodiments, the invention provides a method that includesmounting a plurality of disk-drive connectors in an enclosure, theplurality of disk-drive connectors including at least a first disk-driveconnector and a second disk-drive connector that are each electricallyand mechanically coupled to the enclosure, and mechanically coupling thefirst disk-drive connector and the second disk-drive connector such thatrotational force produced by a first disk drive that is connected to thefirst disk-drive connector is at least partially transmitted astranslational force to a second disk drive that is connected to thesecond disk-drive connector.

In some embodiments, the invention provides a method that includesmounting a plurality of disk drives in an enclosure, the enclosureincluding a connector substrate, the plurality of disk drives includingat least a first disk drive and a second disk drive; vibrationallycoupling the first disk drive to the second disk drive, and sending afirst seek operation to the first disk drive and a second seek operationto the second disk drive, wherein a timing of the first seek operationrelative to the second seek operation is adjusted to minimize adversevibrational interaction between the first disk drive and the second diskdrive.

In some embodiments, the invention provides an apparatus that includes adata structure having a plurality of entries, each entry containingvibration-interaction information relative to a read operation occurringon a first disk drive of a pair of disk drives and a seek operationbeing performed on a second disk drive of the pair.

In some embodiments, the invention provides an apparatus that includes amemory, the memory holding vibration-interaction information, aninformation processing unit operatively coupled to the memory to receivethe vibration-interaction information and adjusting a timing of seekoperations to a plurality of disk drives based on the information.

In some embodiments, the invention provides a method that includesmounting a plurality of disk drives in shock mounts in an enclosure and“detenting” the plurality of disk drives against vibration using adisengagable detent device.

In some embodiments, the invention provides an apparatus that includesan enclosure, a substrate held within the enclosure, a plurality ofdisk-drive connectors each coupled mechanically to the substrate, theplurality of disk-drive connectors including at least a first and asecond disk-drive connector, and an over-shock detector operativelycoupled to the enclosure and adapted to detect and store informationregarding one or more over-shock events.

In some embodiments, the invention provides a method that includesanalyzing vibration-interaction between a plurality of disk drives heldin an enclosure and storing information that is based on the analysisinto a data structure.

In some embodiments, the invention provides a method that includesmounting a plurality of disk drives to disk-drive connectors within anenclosure, adhering a resilient sheet across the plurality of diskdrives, and attaching a cover to the resilient sheet.

In some embodiments, the invention provides an apparatus that includes aplurality of disk drives mounted to disk-drive connectors within anenclosure, a resilient sheet (such as a visco-elastic membrane, forexample) across the plurality of disk drives, and a cover.

BRIEF DESCRIPTION OF THE DRAWINGS

Various features and attendant advantages of the present invention willbecome fully appreciated as the invention becomes better understood uponreading the following description and when considered in conjunctionwith the accompanying drawings, in which like reference charactersdesignate the same or similar parts throughout the several views.

FIG. 1 is a perspective drawing of disk drive 120 mounted in aperpendicular-to-the-major-face orientation (e.g., vertical, if themajor face is horizontal) in a disk-drive system 100.

FIG. 2 is a perspective drawing of a storage system 200 with the diskdrives placed in a new physical-layout pattern 250 that enables the diskdrives themselves to serve as the “fins” of a large heat sink.

FIG. 3A is a block diagram of a power supply 300, as used in someembodiments.

FIG. 3B is a block diagram of a power supply 300′, as used in someembodiments.

FIG. 3C is a block diagram of a power supply 300″, as used in someembodiments.

FIG. 4A is a perspective drawing of disk drives 120 and 120′ mounted ina vertical orientation in a disk-drive system 100.

FIG. 4B is a perspective drawing of a pair of disk drives in a Torientation.

FIG. 4C is a perspective drawing of a pair of disk drives in a Yorientation.

FIG. 4D is a perspective drawing of a pair of disk drives in acounter-rotating parallel orientation with their axes of rotationaligned.

FIG. 4E is a perspective drawing of a pair of disk drives in acounter-rotating parallel orientation with their edges aligned.

FIG. 4F is a perspective drawing of a pair of disk drives in acounter-rotating parallel orientation each with its axis of rotationaligned with an edge of the other disk drive.

FIG. 4G is a plan-view schematic of a herringbone configuration 400′with counter-rotating pairs of disk drives.

FIG. 5 is a plan-view schematic of a herringbone configuration 500 withcounter-rotating pairs of disk drives.

FIG. 6A is a plan-view schematic of another herringbone configuration600 with counter-rotating pairs of disk drives.

FIG. 6B is a plan-view schematic of another herringbone configuration601 with counter-rotating pairs of disk drives.

FIG. 7A shows a plan view of yet another herringbone configuration 700of disk drives.

FIG. 7B shows a perspective view of system 700.

FIG. 8A is a perspective drawing of prior-art “high-density”hard-disk-drive (HDD) enclosure systems 81 and 82 as might be mounted ina rack 80.

FIG. 8B is a perspective drawing of a high-density HDD enclosure system810 according to the present invention.

FIG. 8C is a perspective drawing of a high-density HDD enclosure system811 using a herringbone configuration according to the presentinvention.

FIG. 8D is a perspective view that illustrates a perforated support gridfor a plurality of disk drives with ESD-(electro-static dischargeprevention)-coated visco-elastomeric material.

FIG. 8E is a top view that illustrates nesting support grid for aplurality of disk drives with ESD-(electro-static dischargeprevention)-coated visco-elastomeric material.

FIG. 8F is a perspective view that illustrates system 804 having amolded-in connector 819 support for a plurality of drives mounted in avertical orientation.

FIG. 8G is a top view of system 804 of FIG. 8F.

FIG. 8H is top view that illustrates the distribution of temperaturesensors around the inlet manifold 1112, outlet manifold 1114 andbetween-drive spaces 95.

FIG. 8I is a front view that illustrates the status-display grid 816.

FIG. 8J is a perspective view that illustrates a cover-latchingmechanism that seats the drives into their connectors.

FIG. 9A is a perspective view that illustrates a porous display havingLEDs mounted on a screen that has much space for air flow through thedisplays.

FIG. 9B is a perspective view that illustrates an LCD display mounted onthe inlet air dams allowing much space for air flow around the displays.

FIG. 9C is a front-elevation view that illustrates an LCD displaymounted on the inlet air dams allowing much space for air flow aroundthe displays.

FIG. 10 is a blown-up perspective view of a system 1000 of someembodiments having one or more disk-drive systems 1001 operativelycoupled to one or more central processing units (CPU) 1002 and/or one ormore video-streaming units 1003 or some combination thereof.

FIG. 11 is a plan-view block diagram of a data-storage system 1100 ofsome embodiments of the invention that provides a high density enclosurehaving one or more rows of disk drives.

FIG. 12 is a plan-view block diagram of a data-storage system 1200 ofsome embodiments of the invention that uses tapered inlet and outlet airchambers.

FIG. 13 is a plan-view block diagram of a data-storage system 1300 ofsome embodiments of the invention that uses curving tapered inlet andoutlet air chambers.

FIG. 14 is a plan-view block diagram of a data-storage system 1400 ofsome embodiments of the invention that uses curving tapered inlet andoutlet air chambers, and laterally offset paired drives.

FIG. 15 is a plan-view block diagram of a connector circuit card pair1500 used in some embodiments of the invention.

FIG. 16A is a plan-view block diagram of a data-storage system 1600 ofsome embodiments of the invention that provides a high density enclosurehaving four rows of disk drives.

FIG. 16B is a functional block diagram of a circuit 1608 used in someembodiments of system 1600.

FIG. 16C is a functional block diagram of a circuit 1609 used in someembodiments of system 1600.

FIG. 17 is a plan-view block diagram of a data-storage system 1700 ofsome embodiments of the invention that provides a high density enclosurehaving one or more rows of disk drives accommodating a variable numberof disk drives in each row.

FIG. 18 is a perspective-view block diagram of a data-storage system1800 of some embodiments of the invention that provides one or more rowsof disk drives in an upper portion of the enclosure and one or morepower supplies in an adjacent lower portion of the enclosure.

FIG. 19 is an elevation view of a data-storage system 1900 of someembodiments of the invention that provides a high-density enclosurehaving one or more rows of disk drives.

FIG. 20A is an elevation view of a data-storage system 2000 of someembodiments of the invention that provides a high-density enclosurehaving one or more rows of disk drives arranged in coupled pairs ofcounter-rotating disk drives.

FIG. 20B is an elevation view of a data-storage system 2001 of someembodiments of the invention that provides a high-density enclosurehaving one or more rows of disk drives with an adjustable-heightmid-drive vibration damper 2075.

FIG. 20C is an elevation view of a data-storage system 2002 of someembodiments of the invention that provides a high-density enclosurehaving one or more rows of disk drives with a cast-in-placevibration-damper boot 2076.

FIG. 20D is an elevation view of a data-storage system 2003 of someembodiments of the invention that provides a high-density enclosurehaving one or more rows of disk drives with a cast-in-place mid-drivevibration damper 2077.

FIG. 21 is a front elevation view of a data-storage system 2100 of someembodiments of the invention that provides a high-density enclosurehaving one or more rows of disk drives with vertical beam stiffener 2110and optional vibration damper 2122.

DETAILED DESCRIPTION

In the following detailed description of preferred embodiments,reference is made to the accompanying drawings that form a part hereof,and in which are shown, by way of illustration, specific embodiments inwhich the invention may be practiced. It is to be understood that otherembodiments may be utilized and structural changes may be made withoutdeparting from the scope of the present invention. The references torelative terms such as top, bottom, upper, lower, vertical, horizontal,etc., refer to an example orientation such as used in the Figures, andnot necessarily an orientation used during fabrication or use.

Systems and methods to densely package disk drives in an enclosure,while at the same time reducing negative effects on the disk drives thatare due to drive-to-drive interactions, can improve performance,density, reliability, and also reduce manufacturing costs andoperational costs.

Individual disk drives include one or more head-disk assemblies (HDAs)and the electronics for control and data transfer to and from the disks.The HDA includes one or more disks and one or more actuator on which ahead is attached. An actuator to which a head is attached is positionedwithin the disk drive such that the actuator can be rotated about anaxis to selectively position the attached head to a select location onan adjoining disk. Accordingly, data can be retrieved from, or writtento, a specific location on a disk by movement of the actuator toposition the attached head at the specific location on the disk.

System Environment

The present invention provides improved systems and methods to denselypackage the hard-disk drives in an enclosure, while at the same timereducing drive-to-drive vibration interaction. These can improveperformance, density, reliability, and also reduce manufacturing andoperational costs. Each hard-disk drive (HDD, also called “disk drive”or “drive”) includes one or more HDAs and the electronics for controland data transfer to and from the disks.

High-density packaging of HDDs in an enclosure exacerbatesdrive-to-drive vibration interaction problems. With several HDDs,packaged closely together in single enclosure, potentially many doingsimultaneous head-seeks, the vibration interaction problem is greatlyincreased. Previous systems and methods to package HDDs and reducedrive-to-drive vibration interaction involved mechanical stiffening ofthe enclosure and/or lower density packaging options.

Hard-disk-drives are sensitive to vibration. The performance andreliability of a HDD are decreased with vibration. When multiple HDDsare operating within an enclosure, rotational-acceleration vibrationgenerated from the head-seek operation on one HDD can adversely affectthe read/write operations (and possibly head-seek operations as well) onother HDDs. (Note that non-acceleration vibration such as due todisk-spindle wobble, room noise, or fan vibration is generally lessproblematic than acceleration vibration due to actuator seekoperations.) The drive-to-drive rotational-acceleration vibrationinteraction can cause the heads in an HDD to move off track, and thuscause read-data errors and write-data errors. Such errors may result inadditional revolutions to re-locate the data, excessive retries, lostdata, longer head-seek times, slow data access, increase powerconsumption and heat production. Reducing the vibration transferredbetween HDDs can improve HDD performance, density, reliability,manufacturing costs, and/or operational costs.

FIG. 1 is a perspective drawing of disk drive 120 mounted in aperpendicular-to-the-major-face orientation (e.g., vertical, if themajor face is horizontal) in a disk-drive system 100. In someembodiments, a plurality of other drives (up to one-hundred-fifty,one-hundred-ninety-two, two-hundred or two-thousand drives or more) areeach plugged into their respective sockets (or to other suitableconnectors) (e.g., connector 123) that are coupled to connector circuit129 (e.g., in some embodiments, a plurality of insulated conductorscarrying power and signals to and from drive 120) on connector circuitboard or substrate 150. Disk drive 120 includes one or more disks 115that rotate around their axis 117, an actuator 112 that rotates back andforth around its axis 111 to move its head 114 onto a given track 113 ondisk 115. The data is written serially on each track 113 (e.g., asmagnetic domains in the case of magnetic recording disks, or as opticalartifacts in the case of optical disks, or as atomic-force artifacts orother suitable information), so the head 114 must be moved to and kepton track 113 in order to read the data. Any movement of drive 120 thatcauses the drive 120 to have a rotational force 187 around its Z_(R120)center-of-mass axis, or a transitional rotation vibration force, cancause head 114 to be moved off track 113.

Data is organized on the disk drive 120 in serial fashion. This meansthat the data is stored on individual tracks (e.g., track 113) on thedisk 115, which can be exemplified as concentric rings. A head that ispositioned at a constant radius from the center of rotation of the diskis able to read data from a specific track on the disk as the diskturns. This allows data to be stored and retrieved from specific trackson the disk by positioning the head above the specific track. However,if the position of the head is disrupted (i.e., moved off track), thehead is no longer able to read the data from the desired specific trackand must be repositioned. Accordingly, events that cause the position ofthe head to change in an undesired manner disallow proper reading ofdata from a disk and disallow proper writing of data to the disk.Examples of such events include shock to the disk drive, vibrationalforces, torques, and the like.

The time required to find and transfer data on a disk is referred to asthe access time. Access time can be divided into seek time, rotationallatency, and data transfer time. Seek time refers to the time requiredto position an actuator on a track that contains the desired data.Rotational latency refers to the time required for the disk to spin suchthat the desired data on the requested track is under the head 114 ofthe properly positioned actuator 112. Transfer time refers to the timerequired to transfer the data to or from the head 114 on the actuator115 to a location on a track 113 where the data is stored or retrieved(put to use). The rate of data transfer can be altered by placingdifferent portions of the data on different disk drives (this is calledstriping, explained further below). For example, data can be split intoblocks that are stored on two or more disk drives. Different blocks ofdata can then be read from the multiple disk drives in an overlapped orparallel manner and used as needed without having to wait for a singledisk drive to free up. This process allows overall data to betransferred more rapidly than if the data are stored on a single diskdrive. The rate of data acquisition can also be altered by placingmultiple copies of data onto a disk. For example, five copies of thesame data block can be stored on a single track or closely adjacenttracks of a disk to reduce rotational latency as the disk would onlyhave to turn at most one-fifth of a revolution for one of the copies ofthe data to be accessed (one tenth of a revolution on average), ascompared to accessing data on average in one-half revolution for datathat was stored on the disk as a single copy. (Since the location on thetrack where the head starts is random with respect to the location ofthe data, some of the time the head will reach the track exactly at apoint in time that it can immediately access the data (no revolutiontime), and other times it will take a full revolution until the data isin a position to be accessed; thus, on average the rotational latency isgenerally a half revolution is a single copy of the data is used, and ½Nrevolutions if N copies of the data are stored.) Additionally, storingmultiple copies of data on a single track can decrease the time requiredfor data acquisition in the event of a tracking or other recoverableerror, since the rotational latency would be reduced followingrepositioning of the head following the error.

When data is retrieved or written to a disk, a seek operation is usedthat rotates the actuator about its axis and positions the attached headat the track on the disk where the data is to be written or read. Therotation of the actuator arm produces a rotational force, wherein thedisk drive experiences a rotational force in the opposite direction asthe actuator motion. This rotational force can move the disk drive andthus move the neighboring enclosure and cause a neighboring drive tomove. This can cause the track position of the actuator in thatneighboring disk drive to change and if that disk drive is reading orwriting data at the time, it will thereby cause a read or write error tooccur in the neighboring drive.

In conventional disk-drive arrays, the enclosure and the HDA cases arequite heavy in relation to the mass of the actuator. Accordingly, thedisk drives of the disk-drive arrays are less affected by rotationalforces that are transferred from one disk drive doing a seek operationto a neighboring disk drive doing a read or write operation. As the massof the HDA is reduced, the proportional mass of the actuator increases,and the relative rotational force due to the actuator is relativelylarger. In addition, smaller drives allow the enclosure's metal case(which is used to fabricate the disk-drive-array enclosure) to be madethinner and less rigid. The resulting lighter weight can produce lessdamage to the unit if it is dropped. However, the thinner metal can alsoallow a greater amount of rotational or translational force to betransmitted between drives. Generally, moderate translational force isnot a problem, nor is rotational force that does not move the read-writehead (e.g., rotational acceleration around an axis perpendicular to theactuator axis). With increasingly smaller drives and thinner cases, therotational force from a seek operation in one drive has a largerdeleterious affect (i.e., primarily a rotational force that moves a headoff track) that is transmitted to nearby disk drives and that results inthe problems described.

Accordingly, these negative effects of rotational and translationalforce on disk drives are exacerbated by two major trends in the diskdrive and disk array industries. The first of these is the trend towardsmaller and lighter HDA mechanisms. As HDA mechanisms become smaller (asa function of disk diameter), the mass of platters decreases roughly asa function of the square of the platter radius. The mass of disk drivemotors also tends to decrease exponentially as a function of diskdiameter. However the mass of the head actuator tends to decrease onlylinearly, as a function of the length of the actuator. The result isthat as HDA mechanisms become smaller, the mass of the actuator becomesa proportionately larger part of total HDA mass. The non-actuatorportion of total HDA mass acts (beneficially) as an inertial mass (i.e.,a damper of higher frequency vibrations since the heavier mass has alower characteristic frequency) that attenuates rotational force, so theloss of non-actuator mass in proportion to total HDA mass represents agrowing problem in disk arrays.

The second of these is the trend in disk arrays toward larger numbers ofdisk drives per unit of disk enclosure volume. Conventionally, thesedrives are lined up along the narrow front and/or back surface of theenclosure, where the right-angle comers constrain rotation and/orvibration. As disk drives are packaged more densely, they must bemounted interior to the enclosure on the membranes formed by the lowerand/or upper covers, and the effect of inter-drive mechanical couplingand rotational and translational forces to nearby disk drives isexacerbated. With high-density enclosures and random disk accesses, thepossibility of several HDDs generating additive rotational and/ortranslational forces is increased. In addition, the problem is greatlymagnified for HDDs attempting to hold sector tracking while doing readsor writes.

FIG. 2 is a perspective view that illustrates a storage system 200,according to some embodiments of the invention, with the disk drivesplaced in a new physical-layout pattern 250 that enables the disk drives120, 120′, and disk-drive pairs 205, 206, 207, 208, 209, (each havingtwo disk drives 120) and the like, to individually and collectivelyserve as the “fins” of a large heat sink through which air is drawn orpushed in order to remove heat generated by the disk drives and thedriving circuitry connected to use the disk drives. The arrangement ofthe disk drives further creates a plurality of tuned spaces such asinlet manifold 11 12, outlet manifold 1114 and between-drive spaces 95that control air flow from fans 240 to a high degree of precision inorder to increase cooling efficiency. In some embodiments, the staggeredherringbone orientation of HDDs with graduated spacing between diskdrives is to optimize cooling by forcing airflow between the disk drivesand taking into account the increasing temperature of the air as itmoves through the disk drives. Since heat transfer is proportional tothe temperature difference between the air and the drives, and to theamount of air, more air is used where the air temperature is higher andthe temperature difference is less. In some embodiments, system 200 isconnected to one or more processors 89, each coupled to communicateddata to a plurality of disk-drive enclosure systems 201, 202, and/or 203and the like, each having a large plurality of disk drives 120. In someembodiments, two or more power supplies 231, 232 provide redundant powerfor the disk drives 120. In some embodiments, the fans 240 are locatesat a far end of the airflow through the enclosure so they pull airthrough the disk drives and push the heated air out of the cabinet inorder that the heat from the fans is inserted into the air stream afterit has cooled the other components. In some embodiments, the fans 240are accessible and possibly replaceable by the user or service personsat an exterior surface of the enclosure, but enough redundancy isprovided for the disk drives and power so that the system can continueto operate with substantially full functionality even if multipleindividual components fail. Thus, the disk drives can be held in placein the enclosure using visco-elastic adhesive along one or a few edges,reducing weight and virtually eliminating the need for service calls.Further, small DC-to-DC regulated power supplies can be permanentlymounted (e.g., soldered, in order to reduce connector-caused failures)in place, since multiple ones of the power supplies can fail and yet thesystem continues to function fully using the remaining good powersupplies.

Power-supply Description

FIG. 3A shows a disk drive system 201 having a power supply 300, as usedin some embodiments of the invention. Power supply 300 includes a powercrossover and power router configuration that meets the needs of a densebox of disk drives (DBOD). Power supply 231 includes two DC-to-DC powersupplies 231A and 231B.

In some embodiments, each of these uses a AM80A-048L-050F40 model powersupply available from Astec company. In some embodiments, the input tosuch a power supply includes dual 48-Volt DC supply lines with optionalremote-control telecommunications to control the power. In someembodiments, the power modules can take DC input power from 36- to72-Volt DC. One or more of the following features apply to someembodiments of the invention. The PRIMARY and MIRROR notation refers todrives that provide the primary data storage (the primary copy of data)and the mirrored data storage (the other copy or copies of the data). Insome embodiments, there is no difference between primary and mirrorcopies of data, in that all write operations will write to all copies ofthe data, and read operations will only access one of the copies,wherein the selection of which copy is to be read is made on a rotationor alternating basis, or on a basis of which disk drive is not busy withanother operation at the time when the read operation is started. Forexample, if the data are mirrored three ways, three disk drives willeach have a copy of the same data, and when writing, the write data willbe sent to all three disk drives, but when reading, a first readoperation is sent to only the first disk drive, a second read operationis sent to only the second disk drive, and a third read operation issent to only the third disk drive. When a fourth read operation arrives,it would generally be sent to the first disk drive, but if that diskdrive is still busy with the first read operation, the fourth readoperation could be sent to the second or third disk drive if either ofthose were finished with their earlier operations. By spreading the readoperations among all the drives, it is more likely that a drive with therequested data for a particular read request will be available (that thedata is on a drive that is not already busy with another prioroperation).

In some embodiments, “Power Module Redundancy” is provided on the input,(i.e., each disk drive is configured to receive power from each of twoor more DC-to-DC power supplies) wherein if any DC-to-DC power supplyfails, it can be automatically disconnected and the remaining DC-to-DCpower supply or supplies is able to handle the load. Like aircraftengines that have two spark plugs per cylinder, four cylinders, and“crossed over” ignitions for redundancy (e.g., two-way), someembodiments of the invention take a similar approach. In someembodiments, the sources 48V A and 48V B also cross the primary andmirrored boundaries. Dual redundant input (of the 48-volt DC sources)and the crossover configuration provide capability to power both sidesin the event of a single 48V input loss. Each input can power bothsides. In some embodiments, the power modules are made by Astec andprovide less than 100-mV ripple (which is, in some embodiments, arequirement for the disk drives and some other power supplies cannotmeet this), are parallelable, controllable, provide monitor sensors(e.g., voltage, temperature and current), provide high reliability thatis more than one million hours MTBF (mean time between failures),regulatory approvals, and provide four voltage-range options: 18-36 VDC,36-72 VDC, 90-200 VDC, and 180-400 VDC. This allows some embodiments toobtain power simply from AC, for example using a simple rectifier on thefront end. In some embodiments, these power supplies provide anefficiency of 84 percent typical for 5 volts output, and ripple is 50 mVtypical, and maximum 100 mV. In some embodiments, the entire box orenclosure of a plurality of drives is made to be “Hot Box” swappable(i.e., where an entire subsystem box is swapped out while the system isrunning), with just a little more switching to selectably disconnectpower supplies 231 and 232 from their power sources.

In some embodiments, the next section or stage is the “power router.”This is a plurality of high-current, redundant relays (having arelatively low voltage drop at high current as compared to solid-staterelays that have higher voltage drops) that can interconnect with eachother, or switch power around, providing routing (if one should fail).When no power supply has failed, the switches connect a plurality ofpower supplies to each section of disk drives, thus reducing the amountof power that must be supplied by each power supply (e.g., in normalmode, each power supply provides half the power needed, and once a powersupply fails, the other power supply provides all the power for its diskdrives).

The last stage includes the disk drives. In some embodiments, each diskdrive uses 5 volts DC, 5.5 Watts maximum (less than about one amp duringpower up). Lines drawn that “Link” the disk drives indicate which drivesare mirrored, in some embodiments. This provides a data link betweenvarious copies of the mirrored data across different power sources48-volt source A and 48-volt source B. In some embodiments,battery-backed uninterruptible power supplies (UPS) are provided forthese sources. In some embodiments, Astec AM80A modules produce 240Watts at 5-Volts DC, or 40 Amps at 5-Volts DC, for a 48-VDC input. Insome embodiments, a version is used that is pin for pin compatible butmore expensive, BM80A, 300W, 60A, if a design needs more power.

Some embodiments include four rows of forty-eight disk drives for atotal of one-hundred-ninety-two drives. Rows are powered up one row at atime, sequentially over a period of time. When a row is powered on, theforty-eight disk drives may use 5.5 watts each maximum, just on powerup, thus drawing 264 watts maximum for a short period of time. In someembodiments, two of the 240-Watt DC-to-DC power supplies are wired inparallel to provide this power requirement. Some embodiments provideadditional individually activated relay switches, such that fewer diskdrives (e.g., twenty-four at a time) are powered on at any one time. Insome embodiments, two rows are powered on simultaneously, usingdifferent pairs of DC-to-DC power supplies. In some embodiments, a plotof disk-drive power over time at power up shows transient power to bebelow 0.5 amps after 3 seconds, but even if it is 10 or 15 seconds, orsome other value; some embodiments provide a programmable delay betweenthe power up of rows to keep the power draw well within the capabilityof the power supplies.

Sequencer timing and power control, in some embodiments, is simple, easyto develop and inexpensive. Some embodiments use one or more PIC-brandcontrollers (model PIC16F872, an 8-bit high-performance RISC CPUavailable from Microchip Technology Inc., Chandler, Ariz., is used forsome embodiments) that are RISC-based CMOS technology and have aninterface for chip-to-chip communication. In some embodiments, theyprovide temperature sensing and full environmental control. In someembodiments, the controller is made using one of the chip sets (such asmodel VSC7160 12-Port SAS Expander that can run at 1.5 Gbps and 3.0Gbps, and that includes Table Routing and a Serial SCSI Protocol (SSP)engine, or model VSC7151 9-Port Serial Attached SCSI Edge Expander thatcan run at 1.5 Gbps and 3.0 Gbps) from Vitesse, or other suitablecontroller and/or expander chip sets for just-a-bunch-of-disks (JBOD)control.

FIG. 3B is a schematic of a disk-drive data-storage apparatus 204 havinga power supply 300′. In some embodiments, apparatus 204 includes a firstcircuit board 381 and a first plurality of disk-drive connectors 311that are operatively coupled to the first circuit board 381. Theapparatus also includes a first plurality of electrically controlledrelay switches 378 that include a first relay switch 320, a second relayswitch 322, a third relay switch 326, and a fourth relay switch 324. Theapparatus also includes a first plurality of DC-to-DC power supplies 374that includes a first DC-to-DC power supply 312 and a second DC-to-DCpower supply 314 that are operatively coupled to the first circuit board381. In some embodiments, the DC-to-DC power supplies 374 receive anintermediate power voltage. In some embodiments, the intermediatevoltage is about 48 volts. In some embodiments, the plurality ofDC-to-DC power supplies 374 are connected through the first plurality ofswitches 378 to supply power to each one of the first plurality ofdisk-drive connectors 311. The plurality of DC-to-DC power supplies 374provide crossover power to the plurality of switches 378 such that eachone of the plurality of disk-drive connectors 311 is coupled through theplurality of switches 378 to each one of the first plurality of DC-to-DCpower supplies 374. Dual power inputs with crossover power beingdirected through the plurality of switches to a plurality of disk-driveconnectors provide a redundant supply of power to the plurality ofdisk-drive connectors.

In some embodiments, sequencer 368 is operable to control a plurality ofswitches in order to sequentially power up subsets of a plurality ofdisk drives. Use of a sequencer reduces the magnitude of power surgesoccurring within the apparatus. For example, in some embodiments, theapparatus includes a sequencer 368 that is operable to control aplurality of switches 378 in order to sequentially power-up subsets 352and 354 of the first plurality of disk-drive connectors 311. In someembodiments, sequencer 368 first activates (e.g., applies power to therelay coils) only certain switches (e.g., switches 320 and 324) thatsupply power to one subset of the disk drives (e.g., subset 352), and ata slightly later time (e.g., 0.5 seconds to 5 seconds later, dependingon the length of time that the disk drives draw extra power to spin up),sequencer 368 then activates only certain other switches (e.g., switches322 and 326) that supply power to one other subset of the disk drives(e.g., subset 354). This reduces the maximum power surge that must besupplied by the power supplies 374 and 376 and by the AC-to-DC powersources 370 and 372). In some embodiments, sequencer 368 later activatesonly certain other switches (e.g., switches 328 and 332) that supplypower to one other subset of the disk drives (e.g., subset 356), andstill later sequencer 368 then activates only certain other switches(e.g., switches 330 and 334) that supply power to one other subset ofthe disk drives (e.g., subset 358). At four still later sequentialtimes, sequencer 368 will successively activate the relay switches336-350 to power on subgroups 360, 362, 364, and 366. By dividing thedisk drives into subgroups (e.g., eight subgroups in the embodimentdescribed above), the power surge for spin up is quite reduced.

In some embodiments, either individual power supply 312 or 314 alone canprovide enough power for all of the disk-drive connectors to which it isoperatively coupled. Accordingly, if a power supply 314 fails, theredundant power supply 312 is able to provide power to the plurality ofdisk-drive connectors and the apparatus will continue to operate. Powersupplies that can be used within an apparatus of the invention can beobtained commercially (e.g., ASTEC POWER, Carlsbad, Calif. 92008). Insome embodiments, each power supply will provide less than 100 mV ofripple. In some embodiments, each power supply will produce about 50 mVof ripple. Furthermore, power supplies having a variety ofvoltage-ranges may be used in various embodiments. In some embodiments,an AC power supply is used that has a simple rectifier and avoltage-range of, for example 18-36VDC, 36-72VDC, 90-200VDC, 180-400VDC,or the like. In some embodiments, each power supply within an apparatusis “Hot Box” swappable which enables the power supply to be removed andreplaced while the apparatus is running.

In some embodiments, the apparatus includes one or more AC-to-DC powersupplies or sources 370, 372 that are operable to receive AC wall powerand to generate an intermediate power voltage. In some embodiments, anintermediate power voltage ranges from about 18 volts to about 36 volts.In some embodiments, an intermediate power voltage ranges from about 36volts to about 72 volts. In some embodiments, an intermediate powervoltage ranges from about 90 volts to about 200 volts. In someembodiments, the intermediate voltage is about 48 volts of directcurrent.

In some embodiments, the voltage output from a power supply into eachone of the switches 320 to 350 is a voltage that is suitable to be useddirectly by a disk drive 120 that is plugged into one or more of theplurality of disk-drive connectors 126. Examples of voltages that aresuitable to be used directly by a disk drive include those within arange of 5 volts plus or minus five percent (e.g., for disk drives usingthe industry standard 2.5-inch form factor). In some embodiments, thesuitable voltage is within a range of 3.3 volts plus or minus fivepercent (e.g., for disk drives using the industry standard 1.8-inch formfactor). In some embodiments, the suitable voltage is some othersuitable voltage selected for the disk drives used.

In some embodiments, a first switch 320 is connected to couple a firstDC-to-DC power supply 312 to a first subgroup (proper subset) 352 of afirst plurality of disk-drive connectors 311, and the second switch 322is connected to couple a second DC-to-DC power supply 314 to a secondproper subset 354 of the first plurality of disk-drive connectors 311.

In some embodiments, an apparatus includes a third switch 326 that isconnected to couple a first DC-to-DC power supply 312 to the secondproper subset 354 of the first plurality of disk-drive connectors 311,and a fourth switch 324 that is connected to couple the second DC-to-DCpower supply 314 to a first proper subset 352 of the first plurality ofdisk-drive connectors 311.

In some embodiments, an apparatus includes a fifth switch 332 that isconnected to couple the first DC-to-DC power supply 312 to a thirdproper subset 356 of the second plurality of disk-drive connectors 313,a sixth switch 330 that is connected to couple the second DC-to-DC powersupply 314 to a fourth proper subset 358 of the second plurality ofdisk-drive connectors 313, a seventh switch 334 that is connected tocouple the first DC-to-DC power supply 312 to the fourth proper subset358 of the second plurality of disk-drive connectors 313, and the eighthswitch 328 is connected to couple the second DC-to-DC power supply 314to a third proper subset 356 of the second plurality of disk-driveconnectors 313.

In some embodiments, an apparatus includes a third DC-to-DC power supply316. In some embodiments, an apparatus includes a fourth DC-to-DC powersupply 318.

In some embodiments, an apparatus includes a ninth switch 336 that isconnected to couple a third DC-to-DC power supply 316 to a fifth propersubset 360 of a third plurality of disk-drive connectors 315. In someembodiments, an apparatus includes a tenth switch that is connected tocouple a fourth DC-to-DC power supply 318 to a sixth proper subset 362of the third plurality of disk-drive connectors 315. In someembodiments, an apparatus includes an eleventh switch 342 that isconnected to couple a third DC-to-DC power supply 316 to a sixth propersubset 362 of a third plurality of disk-drive connectors 315. In someembodiments, an apparatus includes a twelfth switch 340 that isconnected to couple a fourth DC-to-DC power supply 318 to a fifth propersubset 360 of a third plurality of disk-drive connectors 315.

In some embodiments, an apparatus includes a thirteenth switch 348 thatis connected to couple a third DC-to-DC power supply 316 to a seventhproper subset 364 of a fourth plurality of disk-drive connectors 317. Insome embodiments, an apparatus includes a fourteenth switch 346 that isconnected to couple a fourth DC-to-DC power supply 318 to an eighthproper subset 366 of a fourth plurality of disk-drive connectors 317. Insome embodiments, an apparatus includes a fifteenth switch 350 that isconnected to couple a third DC-to-DC power supply 316 to an eighthproper subset 366 of a fourth plurality of disk-drive connectors 317. Insome embodiments, an apparatus includes a sixteenth switch 344 that isconnected to couple a fourth DC-to-DC power supply 318 to a seventhproper subset 364 of a fourth plurality of disk-drive connectors 317.

In some embodiments, the apparatus includes a sequencer 368 that isoperatively coupled to each one of the plurality of switches 378, 380,382, and 384 and operable to apply power in a sequence over a period oftime to the plurality of switches 378, 380, 382, and 384 in order toreduce the magnitude of power-on surge.

In some embodiments, the apparatus includes a second circuit board 383to which a second plurality of disk-drive connectors 313 are eachoperably coupled. In some embodiments, an apparatus includes a thirdDC-to-DC power supply 316 and a fourth DC-to-DC power supply 318 thatare both operably coupled to a second circuit board 383.

In some embodiments, the apparatus includes a plurality of disk drivesconnected to a first plurality of disk-drive connectors 311.

In some embodiments, the apparatus is included within an enclosure. Insome embodiments, the enclosure includes a first air-inlet manifold 1112configured to direct air between a first plurality of disk drives and afirst air-outlet manifold 1114 configured to receive warmed air anddirect the warmed air out of the enclosure.

In some embodiments, an apparatus includes a multiprocessor having twoor more processing units and a memory coupled to the processing units,wherein the memory is operable to send and receive data from a firstplurality of disk drives.

In some embodiments, an apparatus includes a video-streaming subsystem,the video-streaming subsystem including one or more processing units anda memory coupled to the one or more processing units and operable tosend and receive data from the first plurality of disk drives and tosimultaneously output a plurality of video streams.

In some embodiments, an apparatus includes a video-on-demand controlleroperable to receive requests for video programming from each one of aplurality of users, and to access and direct video output to theplurality of users based on the requests.

In some embodiments, the invention provides a method that includesoperatively coupling a first plurality of disk-drive connectors 311 to afirst circuit board 381, operatively coupling a first plurality ofDC-to-DC power supplies 374 to the first circuit board 381, andconnecting the DC-to-DC power supplies 374 through a first plurality ofelectrically controlled relay switches 378 to supply power to each oneof the first plurality of disk-drive connectors 311. The plurality ofpower supplies 374 provide crossover power to the plurality of switches378 such that each one of the plurality of disk-drive connectors 311 iscoupled through the plurality of switches 378 to each one of the firstplurality of DC-to-DC power supplies 374. In some embodiments, the firstplurality of electrically controlled relay switches 378 includes a firstswitch 320 and a second switch 322. In some embodiments, the DC-to-DCpower supplies 374 receive an intermediate power voltage. In someembodiments, the intermediate voltage is about 48 volts of directcurrent. In some embodiments, the first plurality of DC-to-DC powersupplies 374 includes a first DC-to-DC power supply 312 and a secondDC-to-DC power supply 314.

In some embodiments, the method includes operatively coupling asequencer 368 to control a first plurality of switches 378 in order tosequentially power up a first proper subset 352 and a second propersubset 354 of a first plurality of disk-drive connectors 311 over aperiod of time.

In some embodiments, the method includes providing an AC-to-DC powersupply 370 that is operable to receive AC wall power and to generate anintermediate power voltage.

In some embodiments, the method includes providing an AC-to-DC powersupply 370 having an intermediate voltage that is about 48 volts ofdirect current. In some embodiments, the voltage output from theAC-to-DC power supply 370 into each one of the switches 378 is a voltagesuitable to be directly used by a disk drive that is plugged into one ormore of the plurality of disk-drive connectors 311.

In some embodiments, the method includes connecting a first switch 320to couple a first DC-to-DC power supply 312 to a first proper subset 352of a first plurality of disk-drive connectors 311, and connecting asecond switch 322 to couple a second DC-to-DC power supply 314 to asecond proper subset 354 of a first plurality of disk-drive connectors311. In some embodiments, the method includes connecting a third switch326 to couple a first DC-to-DC power supply 312 to a second propersubset 354 of a first plurality of disk-drive connectors 311, andconnecting a fourth switch 324 to couple a second DC-to-DC power supply314 to a first proper subset 352 of a first plurality of disk-driveconnectors 311.

In some embodiments, the method includes connecting a fifth switch 332to couple a first DC-to-DC power supply 312 to a third proper subset 356of a second plurality of disk-drive connectors 313. In some embodiments,the method includes connecting a sixth switch 330 to couple a secondDC-to-DC power supply 314 to a fourth proper subset 358 of a secondplurality of disk-drive connectors. In some embodiments, the methodincludes connecting a seventh switch 334 to couple a first DC-to-DCpower supply 312 to a fourth proper subset 358 of a second plurality ofdisk-drive connectors 313. In some embodiments, the method includesconnecting an eighth switch 328 to couple a second DC-to-DC power supply314 to a third proper subset 356 of a second plurality of disk-driveconnectors 313.

In some embodiments, the method includes connecting a ninth switch 336to couple a third DC-to-DC power supply 316 to a fifth proper subset 360of a third plurality of disk-drive connectors 315. In some embodiments,the method includes connecting a tenth switch 338 to couple a fourthDC-to-DC power supply 318 to a sixth proper subset 362 of a thirdplurality of disk-drive connectors 315. In some embodiments, the methodincludes connecting an eleventh switch 342 to couple a third DC-to-DCpower supply 316 to a sixth proper subset 362 of a third plurality ofdisk-drive connectors 315. In some embodiments, the method includesconnecting a twelfth switch 340 to couple a fourth DC-to-DC power supply318 to a fifth proper subset 360 of a third plurality of disk-driveconnectors 315.

In some embodiments, the method includes connecting a thirteenth switch348 to couple a third DC-to-DC power supply 316 to a seventh propersubset 364 of a fourth plurality of disk-drive connectors 317. In someembodiments, the method includes connecting a fourteenth switch 346 tocouple a fourth DC-to-DC power supply 318 to an eighth proper subset 366of a fourth plurality of disk-drive connectors 317. In some embodiments,the method includes connecting a fifteenth switch 350 to couple a thirdDC-to-DC power supply 316 to an eighth proper subset 366 of a fourthplurality of disk-drive connectors 317. In some embodiments, the methodincludes connecting a sixteenth switch 344 to couple a fourth DC-to-DCpower supply 318 to a seventh proper subset 364 of a fourth plurality ofdisk-drive connectors 317.

In some embodiments, the method includes operatively coupling asequencer 368 to each one of a plurality of switches 378, 380, 382, and384 that are operable to apply power in a sequence over a period of timeto the plurality of switches 378, 380, 382, and 384 in order to reducethe magnitude of power-on surge.

In some embodiments, the method includes operably coupling a secondplurality of disk-drive connectors 313 to a second circuit board 383,and operably coupling a third DC-to-DC power supply 316 and a fourthDC-to-DC power supply 318 to the second circuit board 383.

In some embodiments, the method includes including the apparatus 300within an enclosure. In some embodiments, the enclosure forms a firstair-inlet manifold 1112 configured to direct air between a firstplurality of disk drives and a first air-outlet manifold 1114 configuredto receive warmed air and direct the warmed air out of the enclosure.

In some embodiments, the method includes providing a multiprocessor thatincludes two or more processing units and a memory coupled to theprocessing units and that is operable to send and receive data from afirst plurality of disk drives.

In some embodiments, the method includes providing a video-streamingsubsystem that includes one or more processing units and a memorycoupled to the one or more processing units that are operable to sendand receive data from a first plurality of disk drives and tosimultaneously output a plurality of video streams.

In some embodiments, the method includes providing a video-on-demandcontroller operable to receive requests for video programming from eachone of a plurality of users, and to access and direct video output tothe plurality of users based on the requests.

FIG. 3C is a schematic of a disk-drive data-storage apparatus 204″having a power supply 300″. In some embodiments, apparatus 204″ includesa first plurality of disk-drive connectors 311 that are operativelycoupled to a circuit board. The apparatus also includes a firstplurality of electrically controlled voltage regulators 312″-314″ thatare controlled by power-up sequencer 368 and connected to provideredundant sources of operating voltage to disk drives 120 in thesubgroup of disk drives connected to connectors 311. The apparatus alsoincludes a second plurality of electrically controlled voltageregulators 316″-318″ that are operatively coupled to provide redundantsources of operating voltage to disk drives 120 in subgroup 315. In someembodiments, the electrically controlled voltage regulators 312″-318″receive DC power from one of a plurality of sources 388 of anintermediate power voltage. In some embodiments, the intermediatevoltage is about forty-eight volts.

FIG. 4A is a perspective drawing of disk drive 120 mounted in aperpendicular-to-the-major-face-of-the-enclosure orientation in adisk-drive system 400. This disk drive 120 is as described for FIG. 1above.

In some embodiments (as shown in FIG. 4A), a second disk drive 120′ ismounted face-to-face, substantially parallel to, and adjacent to drive120, such that if simultaneous seek operations are performed to bothdrives from the same starting position and to the same ending track, thetwo rotational accelerations will at least partially cancel. Withrespect to drive 120′ and its Z_(R120′) center-of-mass axis (in someembodiments, Z_(R120′) is collinear with, and in the opposite directionas, Z_(R120)), accelerations 147 around its Z_(R120′) center-of-massaxis are in the opposite direction (clockwise versus counterclockwise)and approximately equal in magnitude as accelerations 187 of drive 120.

Rotational and translational forces that are produced by a disk drivecan be transmitted to other disk drives. For example, if the frontcorner 119 (the corner furthest from actuator axis 111) is moved orrotated downward (as a result of torque 192) relative to the rest of thedrive (and/or corner 121 is moved relatively upward), the actuator 112will rotate in a direction 191 taking the head 114 off its track 113.Conversely, if the actuator 112 rotates in a direction 191 for its seekoperation, the front corner 119 moves downward 192 relative to the restof the drive, transmitting rotational force to other drives in itsneighborhood. Moving a head off track during a read or write operationcauses a loss in performance, since an entire disk revolution is neededto get back to the data that was missed when the head moved off track.

Disk drives can be arranged through use of the methods of the inventionto reduce transmission of rotational forces to neighboring disk drives.Additionally, the invention provides multiple disk drives that arearranged within an apparatus so that transmission of rotational forcesfrom one disk drive to a neighboring disk drive is reduced. In someembodiments of the invention, a second drive 120′ is placed back-to-backto drive 120, such that its disks 145 are rotating in the oppositedirection as disks 115, and its actuator 142 moves in the oppositedirection around its axis 141 as does actuator 112 relative to anoutside frame of reference. In some embodiments, connector 116 of drive120 is plugged into socket 126 on board 150, and is held by one or morevisco-elastomeric (or, in some embodiments, elastomeric, rubbery, softplastic or otherwise compliant to some degree) holder(s) 127 and 128.Similarly, connector 156 of drive 120′ is plugged into socket 166 onboard 150, and is held by one or more visco-elastomeric (or, in someembodiments, elastomeric) holder(s) 167. In some embodiments, drives 120and 120′ are mounted so that their Z_(R) center-of-mass axes arealigned, and actuators 112 and 142 are driven with substantiallysimultaneous operations, in order to cancel some or all of therotational force due to their respective seek operations.

In contrast to rotational forces, an up or down movement of board 150 atlocation 118 directly under the drive's center of rotational mass willmerely cause a translation motion in the Y_(T) direction 182, which doesnot cause a rotation around the Z_(R) center-of-mass axis, and thus doesnot cause tracking errors in drive 120. Thus, a rotational forcereceived at point 118 causes fewer problems than if at corner 119 orcorner 121 of drive 120. Further, if the actuator 112 moves in adirection 191 for its seek operation, the point 118 does not move upwardor downward, but experiences a minor twist, transmitting very littlerotational force to other drives if their corner 119 is closest to thispoint 118 on drive 120. Thus, very little rotational force istransmitted from point 118; this causes fewer problems to neighboringdrives if their corner 119 or corner 121 is closest to this point 118.

Translational displacements 180 which move the entire drive 120 in X_(T)direction 181, Y_(T) direction 182, or Z_(R) direction 183 generally donot cause tracking errors, nor does rotational acceleration 185 aroundthe X_(R) center-of-mass axis or rotational acceleration 186 around theY_(R) center-of-mass axis. However, a rotational acceleration 187 aroundits Z_(R) center-of-mass axis is problematic, as described above.

Disk drives are generally able to function adequately in environmentsthat induce/transmit translational vibrations along 3 axes of the drive(translational movements along X_(T), Y_(T) and Z_(T) will not move thehead off track, since the actuator is generally quite balanced on itsrotational axis) and angular acceleration or rotational force about 2axes (X_(R), Y_(R); see FIG. 4A) also do not generally move the head offtrack. However, rotational force that is transmitted to the head-diskassembly (HDA) around the Z_(R)-axis is problematic. Rotation of theactuator around this axis is what moves the head that is attached to theactuator from track-to-track. When caused by the actuator motor, thismoves the head to the desired track during a seek operation. However,when its neighboring drives transmit rotational force to a drive,sector-tracking problems can occur. Even a very small amount ofrotational force is known to increase the position-error signal of thehead, cause instability in the servo system, degrade I/O performance,increase power consumption and increase error rates of disk drives.During any seek operation, an HDA using a rotary head actuator generatesrotational force in a direction opposite to that of the acceleration ofthe head actuator, and transmits this energy to the environment aroundit, including other disk drives. Disk drives are most sensitive torotational force during the sector-tracking media transfer phase ofoperation, but are less sensitive to rotational force during a seekoperation.

The following aspects and embodiments of the invention are aimed atreducing the effects of rotational and translational forces among aplurality of disk drives mounted in a mechanical enclosure. In addition,where RAID hardware or software logic is used to increase theperformance and/or reliability of a plurality of disk drives, thefollowing aspects and embodiments also describe how the disk drives canbe arranged mechanically in relation to one another and in relation toRAID striping and mirroring logic to reduce the effects of rotationaland/or translational forces.

Embodiment A1—Counter-Rotating Disk Drives in a Mirrored Set to OffsetRotational Acceleration Vibration (RAV)

“Mirrored disks” are a set of M (where M is two or greater) disk drivesthat are logically connected as a set and at least some of the datawritten to that logical set is replicated to each of the M drives foreach write operation. In some embodiments, all data sent to the set ofdrives is replicated, while in other embodiments, some amount (e.g.,one-hundred-fifty GB) or some percentage of the drive's data space(e.g., fifty percent) is mirrored and the remaining data on each driveis unique or different, in order to provide mirrored speed andredundancy for the portion that is replicated, while also providing alower cost per gigabyte for the other data by writing only a singlecopy. The processor elements (PEs) or operating system (OS), in someembodiments, could see a set of four three-hundred-GB drives as onefour-way-mirrored drive of one-hundred-fifty GB, plus four non-mirroreddrives of one-hundred-fifty GB each. In some embodiments, some portionor percentage of the data is replicated with a higher number of copies(e.g., a set of four three-hundred-GB drives could have thirty percentof the data or 90 GB replicated four times, once for each disk drive,with the operating system software seeing one 4-way-mirrored ninety-GBdrive), while other data is replicated across fewer drives (e.g., ninetyGB replicated twice to a first pair of drives, and another 90 GBreplicated twice to a second pair of drives, so the OS sees two2-way-mirrored ninety-GB drives), and/or split differently (e.g.,one-hundred-twenty GB replicated thrice to three drives, and anotherone-hundred-twenty GB sent as non-mirrored data to a fourth drive, sothe OS sees one 3-way-mirrored 120-GB drive and one non-mirroredone-hundred-twenty-GB drive). In such mirrored embodiments, everyfull-mirrored write operation is sent to all N drives, so every drivehas a copy of all the data, while subset-mirrored writes are sent onlyto the specified subset. In some embodiments, each of a plurality ofsubsets of the drives have drives placed alternately back-to-back orfront-to-front, as shown in FIG. 4A, so that half of the drives arerotating in the opposite direction as the other half. In someembodiments, read operations are also sent to all N drives (or to all ofthe subset of drives having the replicated data), so the drive that canreturn the data fastest has its data used, and the other drives' data isdiscarded. This provides the increased reliability of the duplicateddata, and increases read performance to that of the drive that happensto have the least rotational latency (by the happy chance of having therotational angle of its disks closest to the requested data) to reachthe requested data. Further, since all seek operations (reads andwrites) are sent to all M drives (or subset of M drives) of the set atsubstantially at the same time, the rotational accelerations of the Msimultaneous seek operations cancel, at least to some extent. Furtherthere are no seek operations for some of the drives while others of thisset of M drives are reading or writing, tracking errors due to RAV arereduced.

Embodiment A2—Counter-Rotating Disk Drives in a Mirrored Set to at LeastPartially Offset RAV, Optionally also Using Read Splitting

Again, every write operation is sent to all M drives, so every drive hasa copy of all the data. In some embodiments, each read operation is sentto only one of the M drives, so the other drives have less utilizationand can accept read operations to retrieve other data. This provides theincreased reliability of the duplicated data, and increases readperformance since more drives can be performing separate read operationssimultaneously. Again, the drives are placed alternately back-to-back orfront-to-front, as shown in FIG. 4A, so that half of the drives arerotating in the opposite direction as the other half. Since all writeseek operations (only for writes) are sent to all M drives of the set,the rotational accelerations of the write-seek operations cancel, atleast to some extent. Further, to the extent that probability allows,the read-seek operation to one drive will not occur during theread-data-tracking portion of a read to another drive of the set of Mdrives. Since all drives have the same data, four successive readcommands to any of the data can each be sent to a different drive.

In some embodiments, read operations to large blocks of data are brokeninto smaller read commands, each to a different portion of the data, andeach sent substantially simultaneously to a different drive of the set.Thus, if M=4, a read operation to fetch, for example, a 640-KB block ofdata is broken into four 160-KB read operations, each sent substantiallyat the same time to a different drive of the set. Thus, four seeks ofsubstantially the same duration and to approximately the same locationson each drive will occur at about the same time. Two would have aclockwise acceleration and the other two would be counter-clockwise. Thefirst drive would return the first 160-KB portion of the 640 KB-readrequest, the second drive would return the second 160-KB portion, thethird drive would return the third 160-KB portion, and the fourth drivewould return the fourth 160-KB portion. This provides the advantage ofthe counter-rotating seek commands canceling some of the RAV, the seekoperations occurring when the other drives are not trying to keep ontrack and not occurring when heads are trying to stay on track, and thespeed of parallel data retrieval providing improved performance.

Some embodiments use vulnerability mapping, described below, as onebasis for selecting which drive or drives are to be used for aread-split read (i.e., a read operation that could be satisfied by datastored on any one of a mirrored set of drives since all prior writeoperations replicated their data on all drives of that mirrored set).

Embodiment A3—Counter-Rotating Disk Alternation in a Striped Set toOffset RAV

“Striped disks” are a set of N disk drives that are logically connectedas a set and data written to that logical set is spread across the set.At some level of granularity, a block of data is broken into sub-blocks,wherein each successive sub-block is written to a different drive. Thus,the block need not wait to be entirely written to or read from one drivein a serial manner, but instead the set of drives works in parallel,each writing or reading their portion of the block. The set of stripeddisks are, or can be, viewed by the system's processors as a singlelogical disk drive having a capacity that is the sum of the capacitiesof all drives in the set, and wherein each successive block of data(where a block can be any convenient size, such as 512 bytes, 8192bytes, or any other desired size) is written to a different drive (witha plurality of N drives, every Nth block is written to the first drive,every N+1^(st) block is written to the second drive, and so on). Whendata is written to or read from the logical disk that includes the setof striped physical disk drives, a single I/O request to the logicaldisk frequently spans two or more logically adjacent physical diskdrives (each having one stripe of the data), and as a result, there is ahigh probability of simultaneous actuator-seek movements among theseneighboring head-disk assemblies (HDAs).

In some embodiments, the minimum processor-level block size is made tobe an N multiple of the minimum drive-level block size, such that everyprocessor-level read or write is automatically striped across N drivesof a set. For example, if N=4 and the processor-level block size is made8 KB, the drive-level block size is made 2 KB, and each read operationfrom the processor causes a read operation to each one of the N drives.An 8-KB processor read causes four 2-KB read operations, while a 16-KBprocessor read causes four 4-KB read operations, one to each of thedrives of the set. A 56-KB write operation causes four 14-KB writeoperations, one to each of the drives of the set. The N operations, oneto each one of the N drives in a set will be to the same logical addresson each drive, and thus cause substantially simultaneous seekaccelerations that cancel if the drives are alternately clockwise andcounterclockwise. Since the N operations that are sent to the N driveseach access 1/N of the data, the data-transfer phase is shortened.Often, the seek accelerations are rotational accelerations that tend tobe substantially simultaneous, and similar in duration,speed/acceleration, direction and frequency in the N drives of a set. Byalternating the position (face-to-face or back-to-back) of each of thedrives in the stripe, the combined rotational accelerations of the HDAswill, by design, offset one another other as shown in FIG. 4A. In someembodiments, for example, a 16-KB read operation from the system isbroken into two 8-KB operations to logically adjacent drives in a set,where a first drive has a seek in a clockwise direction as seen from itstop cover, and a second drive also has a seek in a clockwise directionas seen from its top cover, but when the top covers are face-to-faceadjacent, these two rotational accelerations are in opposite rotationaldirections and at least partially cancel each other's mechanicalmotions. Coupling disk drives in this way, both mechanically and also tothe RAID striping logic, takes advantage of the simultaneity of seekoperations, during which time the disk drives are largely insensitive toRAV disturbances.

In some embodiments, it also takes advantage of the local stiffness ofboard 150 between two adjacent HDD connectors. That is, in someembodiments, the connectors themselves provide stiffening, and in someembodiments, the connectors are molded in pairs such that they are morerigid to one another. In some embodiments, such a pair of unitary-moldedconnectors is attached to the bottom metal plate using a visco-elasticmaterial that dampens any vibrations that otherwise would be transferredto the bottom metal cover, and the more rigid connection between the twoconnectors allows the counter-rotating accelerations to cancel. In someembodiments, more than two drives are alternately placed face-to-face,then back-to-back, then face-to-face, etc., and more than two of thedrives will have simultaneous seek operations (e.g., two clockwisedrives and two counter-clockwise drives).

In other embodiments, a pair of drives has one of its drives write datafrom the inner diameter to the outer, and the other drive writes itsdata from the outer diameter to the inner. Such a pair can be placedboth facing the same direction (i.e., face-to-back) such that when onedoes a seek operation from the inner-to-outer diameter, the other willhave a seek operation from the outer-to-inner (the opposite rotationaldirection), and their total rotational acceleration will at least tosome extent cancel. These embodiments, however, have non-symmetricalseeks at the outer or inner diameters (when one drive seeks at its outerdiameter, the other drive of that pair seeks at its inner diameter),since a seek operation that moves across, for example, 20-GB of data hasfewer tracks to move at the outer diameter than at the inner diameter.

In other embodiments, each pair of drives includes one drive thatrotates its disks and actuator in the opposite rotational direction asthose in the other drive. This requires non-standard drives (i.e., halfof the drives are built as mirror-images as viewed from the cover), butallows all the drives to be facing in the same direction (i.e.,face-to-back). These embodiments, however, have symmetrical seeks at theouter or inner diameters, since a seek operation done at the outerdiameter of one drive, will be accompanied by a seek operation in theopposite direction but also at the outer diameter or the other drive.

Thus, some embodiments have one or more pairs of disk drives, each pairoperated such that their actuators are operated substantiallysimultaneously. This provides the advantage that the counter-rotatingrotational accelerations, at least to some extent, cancel one another.This canceling reduces the RAV transmitted to other drives near the pairin the enclosure, as well as reducing the RAV within the pair. It alsoprovides the advantage, that even if not performed exactly at the sametime, each acceleration (due to an actuator seek) occurs when theneighboring drive is also in or temporally near the seek mode time, andthus is less susceptible to read or write errors than if the rotationalacceleration occurred while that drive's head was on track and trying towrite or read.

In addition, since the system sends down a set of one or moresystem-sized blocks, and the set of N drives (where N can be two or moredrives) each write a disk-sized blocks, each being the system block sizedivided by N, the data can be written twice as fast, once the drivesreach the desired data location (i.e., after the seek and rotationaldelay). Suppose the system has a granularity of 8192 bytes (commonlycalled 8-KB blocks), and the drives are organized as 2-KB blocks (2048bytes), then a pair of drives can write the first and third 2-KB blocksto the first drive of a pair, and the second and fourth 2K the seconddrive. Alternatively, a “pair” could include two physical pairs, or fourdrives, each receiving a 2-KB portion of each 8-KB write operation. Inother embodiments, other numbers of drives can be used in each set ofdrives.

In other embodiments, a replicated set of drives can be provided,wherein data is M-way mirrored and N-way striped. For example, fifteendisk drives can be configured as a 4-way-mirror, 5-way-stripe set ofdrives (e.g., if 200-GB drives are used, five groups of four mirroreddrives each form a one-terabyte logical drive, where each block of datais replicated four times, and data is striped across the five groups ofmirrored drives). In such embodiments, the mirrored drives can beconfigured to have counter-rotating pairs or quads to cancel at leastsome of their RAV. When mirroring is done with an even number ofreplications (i.e., M=2, 4, 6, etc.), all write operations can be RAVbalanced (the same number of seek commands being sent toclockwise-rotating (CW) drives as to counterclockwise-rotating (CCW)drives).

For systems performing read splitting and if a read command specifiesdata kept on an odd number of drives, or for write operations if M and Nare both odd numbers (e.g., M=3 and N=5), approximately half of thedrives can be configured to rotate in the opposite direction as theother approximately half (e.g., 7 CW and 8 CCW drives). Commands sent tosuch a configuration have almost all of the RAV of the set of drivescancelled by almost balancing (for all but one drive) the clockwise (CW)acceleration with counterclockwise (CCW) acceleration.

In other embodiments, an exactly even number of counter-rotating seekoperations can be sent, even if the data requested (to be read orwritten) is kept on an odd number of drives, by sending one unused seekoperation to another drive of a set—for example to a spare drive (i.e.,one with no system data stored on it, but which is provided in order tobe able to swap for a failed drive in the future, if and when a failureoccurs or is predicted) or to an idle drive (i.e., one that has systemdata on it) (e.g., if 3-way mirroring and 5-way striping were used, andone spare drive was provided for the other 15 drives, for each accessthat accessed an odd number of drives (e.g., 3, 9 or 15 drives), a seekto the spare drive would also be simultaneously sent, but for accessesthat accessed an even number of drives (e.g., 6 of 12 drives), no seekcommand would be sent to the spare drive. In this way, counter-rotatingseeks to the drives would always substantially cancel the rotationalacceleration.

Embodiment B—Orthogonal Placement of Disk Drives as Rotational-ForceMass Dampers

When data is read from or written to a plurality of mechanically coupleddisk drives, each seek operation issued to any disk produces acorresponding rotational force in the head-disk assembly (HDA)mechanism, the energy of which is transmitted to surrounding structureswhich either absorb or transmit that energy. When a subject disk absorbsan RAV component produced by another nearby disk about the Z_(R) axis ofthe subject disk, the negative effects of RAV are maximized. Bypositioning a plurality of disk drives orthogonally to each other, theRAV energy created by one drive may be transmitted to and absorbed bythe mass of nearby orthogonally oriented drives without acting on thesubject drives around their Z_(R) axis, which is the axis of greatestsensitivity.

FIG. 4B shows a pair of drives in a T orientation. Neighboring diskdrive 160 is at a right angle (at or about ninety degrees) to referencedrive 120, with its corner 119 placed nearest to point 118 (under thecenter of rotational mass) of drive 120, then drive 120 acts as aninertial mass that resists the rotational force motion 192′ from theneighboring drive 160, and drive 120 suffers little or no trackingerrors, since the neighboring drive's rotational force 192′ acts as aY_(T) translational movement, not a rotational force for drive 120.Conversely, any rotational force 192 of drive 120 causes the leastmotion at point 118 of drive 120, and thus does not cause rotationaltracking errors in the neighboring drive 160. The moments of inertia ofthe drives will be about axes that are orthogonal to each other. Thisallows the mass of orthogonally positioned disk drives to act as a massdamper for rotational force produced by nearby disk drives and allowsrotational force to be dissipated harmlessly around the X-axis andY-axis of subject disk drives, which is an advantageous situation foreither reading or writing of data. A rotational force 187 of drive 120around its Z_(R120) axis causes little or no movement at point 118, andthus causes no tracking error in drive 160. A rotational force 188 ofdrive 160 around its Z_(R160) axis causes only translational movement atpoint 118 of drive 120, and thus causes no tracking error in drive 120.

FIG. 4C shows a pair of drives in a Y orientation. Neighboring diskdrive 159 is at an oblique angle to drive 120, with its corner 119placed nearest to point 118 (under the center of rotational mass) ofdrive 120, then drive 120 acts as an inertial mass (as in FIG. 4B) thatresists the RAV motion from the neighboring drive 159, and drive 120suffers little or no tracking errors, since the neighboring drive'smotion acts as a Y_(T) translational movement, not a rotational force.In some embodiments, the Z_(R159) axis of drive 159 aligns with the backcorner 121 of drive 120 (Z_(R159) axis passes next to the rear edge ofdrive 120). By having the perpendicular plane containing the Z_(R159)axis also include the rear edge of drive 120, then point 118 of drive159 is closer to rear corner 121 of drive 120 than is either of corner119 or corner 121 of drive 159.

FIG. 4D shows a pair of drives in a counter-rotating parallelorientation with their axes of rotation aligned. That is, Z_(R120) axisof reference drive 120 is co-linear with Z_(R161) axis of neighboringdrive 161. In some embodiments, seek operations are synchronized (bypairing, striping or both), such that rotation force 187 of drive 120 isto some extent simultaneous with and cancels some or all of rotationalforce 466 of drive 161. In some embodiments, each pair of drives 120 and161 is mounted in a shuttle or holder 170 that holds the drives at thenecessary offset 171 to align rotational axis Z_(R161) to rotationalaxis Z_(R120), and to provide a convenient carrying, electrical, and/orcooling holder that can be easily inserted into the disk-arrayenclosure. In some embodiments, such a holder 170 is provided for othersets of two or more drives (such as for FIGS. 4A, 4B, 4C, 4E, or 4F), tomake handling easier.

Thus, in some embodiments, the invention provides one or more driveholders or cages 170 for holding a plurality of counter-rotating diskdrives (e.g., drive 120 and drive 161). In some embodiments, each holder170 holds two drives, one drive rotating in a clockwise direction andthe other rotating counterclockwise. In other embodiments, each holder170 holds more than two drives, where half of the drives are rotating ina clockwise direction and the other half rotating counterclockwise. Insome embodiments, each holder 170 includes a single connector to connectto board 150, and a plurality of connections, one to each of thecontained drives. In some embodiments, each holder 170 includes air-flowopenings and one or more air-deflection vanes 175 to help direct theairflow through the drives in the enclosure. In some embodiments, eachholder 170 is substantially only a wire-frame following an outline ofthe drives, wherein the drives are held in place by one or morevisco-elastic or elastomeric bands 173, or are adhesively affixed towire-frame holder 170.

FIG. 4E shows a pair of disk drives (reference drive 120 and neighboringdrive 162) in a counter-rotating parallel orientation with each of theiredges 119 aligned to edge 121 of the other drive. Even though (at leastfor drives whose centers of rotational mass do not coincide with theX-direction centerline of the drive) the center of rotational mass axisZ_(R162) of drive 162 will not exactly align with the center ofrotational mass axis Z_(R120), at least some of the rotational forcewill cancel if the seek operations overlap completely or to some extent.

FIG. 4F shows a pair of drives in a counter-rotating parallelorientation each with its axis of rotation aligned with an edge of theother drive. In this configuration, rather than trying to cancel theclockwise and counterclockwise rotational forces, the corner 121 of areference (first) drive 120 is placed next to thecenter-of-rotational-mass point 118 of the neighboring (second) drive163 (rotational axis Z_(R120) of drive 120 is aligned to the edge ofneighboring drive 163), and the corner 121 of the second drive 163 isplaced next to the center-of-rotational-mass point 118 of the firstdrive 120 (rotational axis Z_(R163) of neighboring drive 163 is alignedto the edge of drive 120). In other embodiments, corners 119 of eachdrive are aligned next to point 118 of the other drive. Thus, either oneof these two drives can perform a seek while the other is trying to reador write, and transfer little or no rotational force to the other drive.In some embodiments, drive 120 is front-to-front facing drive 163, asshown. In other embodiments, drive 120 is front-to-back to drive 163(the front of both drives facing the same direction. In someembodiments, offset 172 is selected to align corner 121 of each drive tothe center-of-rotational mass 118 of the other drive.

Furthermore, the angle alpha (see FIG. 4C) between the two drives may bea function of the structural environment (transmission path) thatconnects the two drives. The mass, stiffness, shape and properties ofthe connecting structure will offer a “tunable” platform to minimizerotational force effects between drives in a paired and/or rowed set.The angle of offset could vary depending on the structure, and may rangefrom 90 degrees (where the drives are perpendicular) to 0 degrees (wherethe drives are parallel, and, in some embodiments, the parallel drivesare offset in the X direction). In some embodiments, the placement ofdrives is based, at least in part, on a computer simulation of theexpected vibration-transmission and/or standing-wave resonance patterns.In other embodiments, a mock-up is built with movable masses thatrepresent the masses of the disk drives (e.g., actual disk drives areused, in some embodiments), and the masses are iteratively moved,tested, moved again, and tested again, etc., until a satisfactoryresonance pattern is achieved that also provides a suitable air-flowpattern for cooling.

FIG. 4G shows a herringbone configuration 400 with counter-rotatingpairs of drives, e.g., 410, 411, 412, 413, and 414 that are inT-orientations to one another, e.g., 420, 421, and 422. Note that in theT-orientation set 421 that counter-rotating pair 413 has its center ofrotation 430 aligned to the corner 440 of pair 410 that move the most ona seek, and also to corner 444 of drive pair 414. This providescross-wise stiffening at these corners 440 and 444, while also exposingthe least sensitive area 430 (i.e., if the center of rotation (COR) ismoved up or down, there is much less likelihood of error than if thisarea is rotated) of drive pair 413.

FIG. 5 shows, for some embodiments, a herringbone T configuration 500with counter-rotating pairs of drives. The drives 548 at the downstreamend of a heating air flow are spaced further apart than are the drivesat the upstream end. Further, the center area 550 is left open so thesource of cooling air has better access to the drives deep in theenclosure.

FIG. 6A shows another herringbone configuration 600 withcounter-rotating pairs of drives, not in a T-orientation, but in aparallel configuration that places a corner 640 of a first pair ofdrives 601 closest to the COR 612 if a second pair of drives 602, andthe corner 641 of drive pair 602 next to the COR 611 of drive pair 601.This pattern is repeated for pairs 603, 604, 605, 606, and 607, and inthe series of pairs to the right etc. Notice also that the rearward orupward end drives 605, 606, and 607 are spaced further from one anotherthan are the frontward or lower drives 601, 602, and 603. In someembodiments, the fans are omitted and the 692 end of the enclosure isuppermost and the 691 end is lowermost (e.g., of a vertically-alignedenclosure), in order that heat convention pulls air up through theenclosure, allowing cooling with fewer or no fans. In other embodiments,the enclosure is mounted horizontally, with ends 691 and 692substantially horizontally aligned with one another, and fans providingthe air movement. In some embodiments, air “turbulators” 695 areprovided, particularly for the wider spaced, in order to introduceturbulence and have more of the cooling air come into contact with thedrive pairs 605-607.

FIG. 6B is a schematic plan view of the configuration of an enclosure601 having a plurality of approximately right-angled paired disk-driveconnectors 629. In some embodiments, each individual one of thedisk-drive connectors 129 of the first plurality of disk-driveconnectors 650 and each corresponding respective one of the disk-driveconnectors 129′ of the second plurality of disk-drive connectors 651 areoriented so that each pair of connectors form about a ninety-degreeangle. In some embodiments, each pair 629 has a first connector 129 fora first disk drive 120 and a second connector 129′ for a second diskdrive 120′, where a corner of the first disk-drive connector 129 is neara corner of the second disk-drive connector 129′ and the two connectorsare oriented at an approximately ninety-degree angle to each other. Insome embodiments, a first plurality of disk-drive connectors 129 arecoupled electrically and mechanically to a substrate 150 in a first row650 and a second plurality of disk-drive connectors 129′ are coupled tothe substrate 150 in an adjacent second row 651 that is substantially amirror image of the first row 650. In some embodiments, a disk-driveconnector 129 in the first plurality of disk-drive connectors 650 and asecond disk-drive connector 129′ in the second plurality of disk-driveconnectors 651′ are oriented such that a disk drive 120 connected to thefirst disk-drive connector 129 produces a rotational force at theadjacent corner 670 (e.g., downward into substrate 150 for a particularseek direction and magnitude) that is opposite that produced by a seconddisk drive 120 that is connected to the second disk-drive connector 129′at the adjacent corner 671 (e.g., up out of substrate or board 150 for aparticular seek direction and magnitude). In some embodiments, data isstriped across the disk drives 660 that are connected to the firstplurality of disk-drive connectors 650 and the same data is mirrored toand striped at corresponding locations (e.g., logical-block addresses,or LBAs) across the disk drives that are connected to the secondplurality of disk-drive connectors 661. In some embodiments, data thatis striped on disk drives 660 that are connected to the first pluralityof disk-drive connectors 650 is mirrored onto corresponding respectiveones of the plurality 661 of disk drives 120 that are connected to thesecond plurality of disk-drive connectors in row 651, such thatrotational force resulting from a read or write function in the firstplurality of disk drives is opposed by the rotational force resultingfrom the same read or write function in the second plurality of diskdrives. In some embodiments, inlet air dams 618 at the air inlet sideforce air 113 into the inlet manifolds 1112, then between the drives andoutlet air 115 is drawn by fans 696 out the outlet side (e.g., rear) ofenclosure, and outlet air dams 616 form the rest of the airflowguidance. In some embodiments, circuit board 150 is made in two or more(e.g., horizontal) parts 1512 and 1514 that connect to a single (e.g.,vertical) circuit board that is connected to both connectors 513 and515, and provides wiring to either a controller card mounted parallel toboard 150 at the opposite end of drives 120, or to cables running outthe rear of the enclosure (e.g., at the top of FIG. 6B).

FIG. 7A shows a plan view of yet another herringbone configuration 700with alternating counter-rotating pairs of drives 701, 702, 703, 704.The drives at the top or back 692 of the enclosure 692 are spacedfurther apart than are the drives near the bottom or front 691. In someembodiments, the controller board 694 is mounted on edge between thecovers to stiffen them and provide vibration isolation. In someembodiments, a display 1695 (either one-sided or two-sided) is mountedto stick out at a right (or other suitable) angle from the front orbottom of the enclosure, so as not to interfere with air flow throughthe fans 696, while providing easy viewing at an angle for a user infront of the unit.

FIG. 7B shows an abstraction perspective view of storage subsystem 700of FIG. 7A.

FIG. 8A is a front perspective drawing of prior-art “high-density”hard-disk-drive (HDD) enclosure systems 81 and 82 as might be mounted ina rack 80. In some embodiments, enclosure system 81 is 3U or 5.25 incheshigh (13.34 cm), while in others enclosure system 82 is 2U or 3.5 incheshigh (8.89 cm). In some embodiments, enclosure system 81 contains aplurality of disk drive enclosures 91, whereas in other embodimentsenclosure system 82 contains a plurality of drive enclosures 92.

FIG. 8B is a front perspective drawing of a high-density HDD enclosuresystem 810 according to the present invention. In some embodiments, thisenclosure is 4U high or 7 inches (17.78 cm), and contains the pluralityof disk drives 120, each drive 120 coupled to the enclosure 892 via oneor more connectors 110. In some embodiments, a plurality of drives 120is aligned in one or more substantially straight rows 850.

FIG. 8C is a top-down perspective drawing of a high-density HDDenclosure system 811 using a “herringbone” configuration according tothe present invention. This herringbone configuration contains theplurality of disk drive enclosures 92, separated by one or more tunedair-flow spaces such as inlet manifold 1112, outlet manifold 1114 andbetween-drive spaces 95. In some embodiments, system 811 contains one ormore fan 696 for allowing air to flow into the system 811, and one ormore of these fans 696 for urging air to flow out of the system 811.

FIG. 8D is a front perspective view that illustrates a system 812 havinga perforated support grid 819 for a plurality of disk drives 120 with ananti-ESD-coated (i.e., having a high-resistivity (but not insulating)coating for electro-static discharge prevention and/or dissapation)visco-elastomeric material, and height-adjustment screws 820.

FIG. 8E is a top view that illustrates a system 813, which, in someembodiments, includes a set of nesting support grids 818 (for aplurality of disk drives 120) made with ESD-(electro-static dischargeprevention)-coated visco-elastomeric material. In some embodiments, eachsupport grid 818 fits over a pin 817 and provides a plurality ofspaced-apart connection points 821 to each drive 120.

FIG. 8F is a front perspective view that illustrates system 814, which,in some embodiments, has a separate molded-in ESD-coatedvisco-elastomeric material connector support (mold-in connector support)846 for each one of a plurality of drives 120 mounted in a verticalorientation. In some embodiments, each of the drives 120 has a notch 844to independently secure each of the drives 120 in the mold-in connectorsupport 846. This notch 844 locks into a detent 845 in support 846. Eachof the drives 120 connects to a circuit board via a connector 126.

FIG. 8G is a top view of system 814 of FIG. 8F that, in someembodiments, contains a plurality of the illustrated drive 120 eachsecured in its molded-in connector support 846.

FIG. 8H is a top view that illustrates the top view of a high-densityHDD enclosure system 815 using a herringbone configuration according tothe present invention, wherein, in some embodiments, there is adistribution of temperature sensors 851 around the tuned airflow spacessuch as inlet manifold 1112, outlet manifold 1114 and between-drivespaces 95. This herringbone configuration contains the plurality of diskdrives 120, separated by one or more tuned airflow spaces such as inletmanifold 1112, outlet manifold 1114 and between-drive spaces 95. In someembodiments, system 815 contains one or more fans 240 for allowing airto flow into the system 815, and one or more of these fans 240 forallowing air to flow out of the system 815.

FIG. 8I is a front view that illustrates a status display grid system816, wherein, in some embodiments, the display grid system is composedof various light emitting diodes (LED). Specifically, in someembodiments a green LED 861 is used by itself or in combination with ayellow LED 862 and/or a red LED 863. And again, in some embodiments theyellow LED is used by itself of in combination with the green LED 861and/or the red LED 863. In still further embodiments the red LED 863 isused by itself or in combination with the green LED 861 and/or theyellow LED 862.

FIG. 8J is a perspective view that illustrates an exposed front view ofa system 817 wherein, in some embodiments, a cover-latching mechanism isused to seat the drives into their connectors. In some embodiments, thiscover-latching mechanism is contained in a case 852 which is 4U high or7 inches (17.78 cm)., and is placed into a 19 inch (48.26 cm) rack unit.In some embodiments, contained within this case 852 is a plurality ofdrives 120, which can be seated or unseated using a cam 872 mechanismmovably attached to a handle 871. The handle 871 is used to lift orlower the cam 872 and to seat or unseat the plurality of drives 120.When the plurality of drives 120 are seated, the cam 872 sits recessedin a slot 873. In some embodiments, individual drives 120 may be seatedor unseated using the above disclosed cover-latching mechanism.

FIG. 9A is a perspective view of a system 900 that illustrates a porousdisplay 910 having LEDs 911 mounted on a screen 912 that has much spacefor air flow 920 through the display. In some embodiments, the display910 includes a plurality of different color LEDs (e.g., red, green,blue, and/or yellow) that can be activated by control unit 915 thatsenses various parameters in system 900 (such as temperature, air flow,disk-drive status, performance (e.g., input-output operations persecond, or IOPS, and the like), and generates appropriate text and/orgraphical display messages that are transmitted to the array 910 of LEDs911 for viewing by a user or operator. In some embodiments, a connector919 is provided to connect controller unit 915 to the display 910. Byattaching the LEDs to a sparse grid having conducting wires therein, airflow is improved since the air can flow through the display rather thanbeing forced around the display. In some embodiments, a grid is providedhaving openings that are approximately 6 mm by 6 mm passing through agrid having grid support (e.g., wiring and insulating supports) that isabout 1 mm or less in diameter.

FIG. 9B is a perspective view of a system 901 that illustrates one ormore LCD displays 930, 931 mounted on the inlet air dams 918 allowingmuch space for air flow 920 around the displays 930 and 931. Theconfigurations of displays 930 and 931 provide an alternative to theconfiguration of flow-through display 910 of FIG. 9A. In someembodiments, a circuit board 1500 has a plurality of disk-driveconnectors 1923, each of which connects to its respective disk drive120. In some embodiments, the disk drives 120 are mounted to the topside of board 1500, and one or more DC-to-DC power supplies 1866 areattached to the bottom of board 1500. In some embodiments, a pluralityof cross-brace members 941 and 942 are provided between bottom cover1979 and circuit board 1500 to provide stiffness. In some embodiments, acenter circuit board 966 (in some embodiments, board 966 includes one ormore metal I-beams in parallel with it for further stiffness—see FIG.21). In some embodiments, a controller unit 953 includes a controllercircuit board 960 that includes a plurality of serial expander circuits1663, 1665, and a top sheet metal cover 961. In some embodiments,enclosure 950 includes a bottom enclosure 952 that provides an airmanifold for power supply 1866, a middle enclosure 951 that provides airmanifolds 1112 and 1114 directing air around disk drives 120 and a topenclosure 953 directing air around controller card 960. In someembodiments, center board 966 is pulled into a socket on board 1500,and, in turn, provides a plug-and-socket connection 964, 965 tocontroller board 960. In some embodiments, expander circuits 1663, 1665,are distributed among top-controller card 960, middle connector board966, and disk-drive connector board 1500. In some embodiments, diskdrives are arranged in pairs 120, 120′ that are oriented and operated tocounteract rotational vibration, as described elsewhere herein. In someembodiments, fans 1615 mounted on the rear of system 901 pull air 920through the system between the drives 120, across the circuit boards1500, 966, and 960 and around the power supplies 1866. The air isexhausted through outlet ports 1202 and the rear of the unit.

FIG. 9C is a front elevation view of system 901 that illustrates LCDdisplays 930, 931 mounted to the inlet air dams allowing much space forair flow around the displays and between the drives 120. The otherreference numbers indicate features and configurations of thecorresponding units shown in FIG. 9B and described above.

FIG. 10 is an illustration of a system 1000 wherein, in someembodiments, one or more multiple-disk-drive units 901 are operativelycoupled to one or more multi-processors (MPs) 1002, of supercomputer1005 and/or one or more video-streaming unit 1003. In some embodiments,each MP 1002 includes memory 1009 and two or more processing elements(PEs) 1008. In some embodiments, supercomputer 1005 is a highperformance scientific computer well known in the art. In someembodiments, supercomputer 1005 is connected to an internet 50.Video-streaming units 1003, in some embodiments, provide the capabilityfor video-on-demand to a large plurality of subscribers such as homes 55connected to cable system 56, in order to provide each subscriber with aselectable source of television programming.

In some embodiments, the invention includes a computer-readable medium51 (such as a diskette, CDROM, FLASH ROM with a USB plug,internet-connected data source, or the like) having control information(such as, for example, instructions, tables, formulae, statetransitions, data structures, and/or the like) stored thereon forcausing a suitable programmed apparatus, such as system 1000 or othersystem described herein, to execute one or more of the methods describedherein. For example, in some embodiments, supercomputer 1005 and/orvideo-streaming units 1003 of FIG. 10 provides a programmableinformation processor that is coupled to read and obtain controlinformation (such as instructions and/or data structures) fromcomputer-readable medium 51 (which can include storage that is accessedacross internet 50), and coupled to control apparatus 1000 or othersystem described herein, according to the instructions stored on themedium.

FIG. 11 is a plan-view block diagram of a data-storage system 1100 ofsome embodiments of the invention that provides a high density enclosurethat, in some embodiments, has one or more rows 1150 of disk drives 120(only one row 1150 is shown in FIG. 11). In some embodiments, system1100 includes an enclosure 1110 that holds a plurality of disk drives120 in a straight row 1150. Other embodiments provide a plurality ofsuch rows. In some embodiments, enclosure 1110 is fabricated from sheetmetal. In other embodiments, the enclosure is fabricated from othermaterials that include plastic, fiberglass, reinforced composites, andthe like. In some embodiments, enclosure 1110 is made to a standard formfactor such as a five-unit (or 5U, referring to a height) enclosure fora nineteen-inch (48.26 centimeter) rack. (A rack unit or “U” is anElectronic Industries Alliance (EIA) standard unit for measuring theheight of rack-mount-type equipment. One rack unit isone-and-three-fourths inches (1.75 inches) (about 4.45 cm) in height. A5U enclosure is eight-and-three-fourths inches (8.75 inches) (about22.23 cm) high. Enclosure 1110 has a first surface 1138 facing the airinlet side 1101 (the side having inlet port 1109, which is typicallycalled the “front”) and an opposite second surface 1136 facing the airoutlet side 1102 (the side having exit port 1119, which is typicallycalled the “back”). In some embodiments, side 1101 also includes one ormore user-input buttons and/or a status display for showing the statusof the enclosure as a whole, performance numbers, the status of one,several, or all the enclosed disk drives, and the like.

In some embodiments, a plurality of systems 1100 (e.g., two rows, threerows, four rows, or any other number of rows 1150) are enclosed side byside in a single enclosure sharing a common first surface 1138 andsecond surface 1136. In some embodiments, a first face 1121 of each diskdrive is facing one direction along the axis of row 1150 and theopposing second face 1122 is facing the opposite direction along row1150. For example, in some embodiments, the first face 1121 includes ametal cover 1123 that covers the disks and actuator and opposite side(second face 1122) includes a printed circuit card 1124 that holds theelectronics for the disk drive 120. Along one side of disk-drive row1150 is air-inlet manifold 1112 that conveys inlet air 1113 to one edgeof the disk-drives 120 in row 1150. In some embodiments, the plancross-section shape of inlet manifold 1112 is rectangular, and the plancross-section shape of outlet manifold 1114 is also rectangular inshape. Thus, each of the disk drives is aligned along a straight lineperpendicular to the “front” first surface 1138 and to back secondsurface 1136. In some embodiments, a visual display panel is mounted onsurface 1138 to show information messages and/or the status of eachindividual disk drive 120. Along the opposing side of disk-drive row1150 is air-outlet manifold 1114 that conveys outlet air 1115 from theopposite edge of the disk-drives 120 in row 1150. In some embodiments ofthe apparatus 1100, the inlet air manifold 1112 has a length 1141measured parallel to the first row that is longer than the inlet airmanifold's width 1142 measured perpendicular to the first row 1150, andwherein the outlet air manifold 1114 has a length measured parallel tothe first row that is longer than the outlet air manifold's widthmeasured perpendicular to the first row 1150.

In some embodiments, enclosure 1110 is oriented vertically such that thecool inlet air is induced upwards within air-inlet manifold 1112, thenthe cross-face air 1111 flows between each adjacent drive in horizontaldirection and is heated, the warm outlet air 1115 rises by convection tothe exit port 1119 of air-outlet manifold 1114. This convection helpspull additional inlet air into the system 1100. In some otherembodiments, a fan is provided to provide increased air flow. In somesuch embodiments, the fan is positioned at the exit port 1119 ofair-outlet manifold 1114 in order that its self-generated heat (e.g.,about 2 watts for each fan, in some embodiments) is inserted into theairstream as it exits the enclosure, after the air has passed across thedisk drives, thus improving the disk-drive heat-transfer characteristicsof system 1100.

In some embodiments, the spacings 1126 between disk drives increase inrelation to their position in row 1150 from the inlet side (the bottomof FIG. 11) to the outlet side (the top of FIG. 11), in order that anequal amount of cooling is provided to each of the disk drives. Forexample, some embodiments provide a relatively small spacing 1125between disk drives 120 near the air inlet side 1101 and a relativelylarger spacing 1127 between disk drives 120 near the air outlet side1102. In some embodiments, the same small spacing 1125 is used for eachof the disk drives near the air inlet side 1101 and the same largerspacing 1127 is used for each of the disk drives near the air outlet1102 and, intermediate spacing is used for disk drives between. In someother embodiments, a gradually increasing spacing is used (e.g.,following an exponential curve) in which the spacing follows theexponential curve with an increase in spacing occurring toward the airoutlet side 1102.

In some embodiments, the air flow speed and turbulence creates astanding wave of variable pressure and the spacings between individualpairs of the disk drives are empirically determined or varied (otherembodiments use computer analysis of the air flow to adjust thespacings) to compensate for the standing wave and provide more evencooling for each disk drive 120. In some embodiments, the amount ofairflow decreases in relation to the distance from the air inlet andthus the spacing between the drives is increased in order to achieve anequivalent amount of air cooling for each disk drive 120. At the airinlet side, a blocking panel 1118 provides an enclosed airspace at thebottom face (the face closest to the bottom of FIG. 11, which dependingon the orientation of the enclosure 1110, may or may not be downwardfacing in the installed system 1100) of the first disk drive 120 in row1150. A corresponding blocking panel 1116 provides an enclosed air spaceat the top face (the face closest to the top of FIG. 11, which dependingon the orientation of the enclosure 1110, may or may not be upwardfacing in the installed system 1100) of the last disk drive 120.

FIG. 12 is a plan view block diagram of a data-storage system 1200 ofsome embodiments of the invention that uses tapered inlet and outlet airchambers. System 1200 holds a plurality of disk drives 120 in a straightrow 1250 that is oriented in a non-perpendicular acute angle relative tofirst surface 1238. System 1200 has a first surface 1238 facing the airinlet side 1201 and an opposite second surface 1236 facing the airoutlet side 1202. In some embodiments, a plurality of systems 1200(e.g., two rows, three rows, four rows, or any other number of rows1250) are enclosed side by side in a single enclosure sharing a commonfirst surface 1238 and second surface 1236. In some embodiments, a firstface 1121 of each disk drive is facing one direction along the axis ofrow 1150 and the opposing second face 1122 is facing the oppositedirection along row 1250. Along one side of disk-drive row 1250 isair-inlet manifold 1212 that conveys inlet air 1113 to one edge of thedisk-drives 120 in row 1250. In some embodiments, the plan cross-sectionshape of inlet manifold 1212 is substantially triangular, and the plancross-section shape of air-outlet manifold 1214 is also substantiallytriangular in shape. In some embodiments, a visual display panel (notshown), such as an LCD dot matrix display with backlighting or an LEDdot-matrix display, is mounted on front surface of triangle-shaped airblocking structure 1218 in order to be able to show information messagesand/or the status of each individual disk drive 120. Note that due tothe triangular shape of inlet manifold 1212 and the diagonal orientationof row 1250, a much larger (in some embodiments, about twice the area)air inlet port 1201 is provided compared to air inlet port 1101 of FIG.11. Thus the display area on the front of blocking structure 1218 issmaller. Along the opposing side of disk-drive row 1250 is air-outletmanifold 1214 that conveys outlet air 1115 from the opposite edge of thedisk-drives 120 in row 1250.

In some embodiments, system 1200 is oriented vertically such that thecool inlet air 1113 is induced upwards from inlet port 1209 withinair-inlet manifold 1212, then the cross-face air 1111 flows between eachadjacent drive in an upward-angled direction and is heated, the warmoutlet air 1115 rises by convection to the exit port 1219 of air-outletmanifold 1214. This convection helps pull additional inlet air into thesystem 1200. In some other embodiments, a fan is provided to provideincreased air flow. In some such embodiments, one or more fans arepositioned at the exit port 1219 of air-outlet manifold 1214 in orderthat its self-generated heat (e.g., about 2 watts for each fan, in someembodiments) is inserted into the airstream as it exits the enclosure,after the air has passed across the disk drives, thus improving thedisk-drive heat-transfer characteristics of system 1200. Because of thediagonal orientation of the drive, a larger area is available forinstallation of fans or other air-movement devices.

In some embodiments, the spacings between disk drives 120 increase inrelation to their position in row 1250 as described above for FIG. 11.Other aspects of disk drive spacing described for FIG. 11 also apply tosome embodiments of system 1200.

At the air inlet side 1201, a substantially triangular blockingstructure 1218 provides an enclosed airspace at the bottom face (theface closest to the bottom of FIG. 12, which depending on theorientation of the system 1200, may or may not be downward facing in theinstalled system 1200) of the first disk drive 120 in row 1250. Acorresponding blocking structure 1216 provides an enclosed air space atthe face of disk drive 120 closest to the top of FIG. 12.

In some embodiments of apparatus 1200, the inlet air manifold 1212 has alength 1141 measured parallel to the first row that is longer than theinlet air manifold's width 1142 measured perpendicular to the first row1250, and wherein the outlet air manifold 1214 has a length measuredparallel to the first row that is longer than the outlet air manifold'swidth measured perpendicular to the first row 1250. In some embodimentsof apparatus 1200, the inlet air manifold 1212 has a length 1241measured perpendicular to air inlet side 1201 that is longer than theinlet air manifold's width 1242 measured parallel to air inlet side1201, and wherein the outlet air manifold 1214 has a length measuredperpendicular to air outlet side 1202 that is longer than the outlet airmanifold's width measured parallel to air outlet side 1202. In someembodiments, one or more of these conditions also applies to theapparatus shown in FIG. 13, FIG. 14, FIG. 16A, FIG. 17, FIG. 18, andother systems described herein.

FIG. 13 is a plan view block diagram of a data-storage system 1300 ofsome embodiments of the invention that uses curving tapered inlet andoutlet air chambers. System 1300 holds a plurality of disk drives 120 ina curved row 1350 that is oriented relative to first surface 1338.System 1300 has a first surface 1338 facing the air inlet side and anopposite second surface 1336 facing the air outlet side. Along one sideof disk-drive row 1350 is a curved substantially triangular shapedair-inlet manifold 1312 that conforms to the shape of the curve of row1350. In some embodiments, the plan cross-section shape of outletmanifold 1314 is curved to conform to the opposite curved side of row1350. In some embodiments, the curve of row 1350 substantially followsan exponential curve, in order to provide more even air flow betweeneach of the adjacent disk drives. Other aspects of system 1300 are asdescribed above for FIG. 11 and FIG. 12.

FIG. 14 is a plan view block diagram of a data-storage system 1400 ofsome embodiments of the invention that uses curving tapered inlet andoutlet air chambers, and laterally offset paired drives. System 1400holds a plurality of disk drives 120 in a curved row 1450 that isoriented at an angle relative to first surface 1438, however, the diskdrives are arranged in coupled pairs 1430, each pair having a disk drive120 facing generally towards air inlet side 1438 and another disk drive120 facing in an opposite direction (generally towards air outlet side1436). For example, a first disk drive 120 can have its metal face 1121′facing the exit side 1436 and its printed circuit side 1122 facing inletside 1438, while the other drive of the coupled pair 1120′ has its metalface 1121′ facing inlet side 1438 and its printed circuit side 1122′facing outlet side 1436. Thus each coupled pair 1430 includes a diskdrive 120 having disks that rotate in a first direction (for example,clockwise) and another disk drive 120 having disks that rotate in anopposite direction (for example, counterclockwise). More important thanthe direction of disk rotation, in some embodiments, is the direction ofrotational acceleration due to actuator seek operations. This is becausedisk rotation assumes a steady-state velocity (no acceleration due todisk rotation), however actuator seek operations cause rotationalacceleration that can be transmitted as a vibration to neighboring diskdrives. This rotational acceleration vibration can force a transduceroff its desired track during a read or write operation thus causing anerror and a retry or recovery operation which slows the system andhinders performance.

Some embodiments mirror data across a two (or more) drives that arephysically across from one another in adjacent rows of disk drives. Insome embodiments, the data is mirrored across a pair of (i.e., two) diskdrives, wherein each write access writes the same data to the same(corresponding) addresses in each respective disk drive, and whereineach read access is sent to only one drive (either alternating betweenthe two drives, or sent to the drive that is idle at the moment). Byalternating or spreading the read accesses so a read is sent to only onedisk drive of a set, the disk drives are less busy and more available toquickly access the requested data. In some embodiments, the mirroredpair are physically oriented to be perpendicular to one another, or at anon-parallel angle, in order to provide additional stiffness andvibration resistance.

Some embodiments stripe data across multiple disk drives in a row. Insome embodiments, this is done in addition to mirroring as justdescribed. In some embodiments, the system's address space is dividedinto a plurality of stripes, and each stripe is multiple sectors (e.g.,using a plurality of adjacent logical block addresses) located on onedisk drive, and successive stripes are located on different disk drives.For example, in some embodiments, each stripe is the same size (e.g., 32sectors/16 KB, 64 sectors/32 KB, 128 sectors/64 KB, 256 sectors/128 KB,512 sectors/256 KB, 1024 sectors/512 KB, 2048 sectors/1 MB, or othersuitable sizes).

Some embodiments “fork” data across two or more drives. Forking dataacross disk drives is similar to striping data across drives, exceptthat the minimum size of a data access (a read or write operation) bythe system (e.g., one kilobyte, in some embodiments using two diskdrives, or two KB in embodiments using four disk drives) is an integermultiple of the minimum size of a data access (a read or writeoperation) allowed by each drive (e.g., one-half kilobyte, in someembodiments). In some embodiments, every read access and every writeaccess to a forked set of drives causes all drives of the forked set toperform the same access (i.e., since the same access is sent to the sameaddress on each drive, all drives will start and end on the same trackas the other drives. This reduces the number of independent arms, butincreases the data transfer rate while keeping the seek and rotationallatency the same. Further, if a pair of forked drives is physicallyoriented so that the rotational accelerations at least partially cancelbecause of the simultaneous seeks, this can reduce tracking errors andimprove performance for some workloads. For example, in someembodiments, the even numbered sector addresses would be sent to onedisk drive of a mechanically coupled pair, and the odd numbered sectoraddresses would be sent to the other disk drive of the pair. Datatransfer times are thus substantially reduced, especially for long datalengths. By forking the data evenly across a pair of disk drives 120such that half of every data block is on the clockwise rotation diskdrive and the other half of the respective data blocks is on thecounterclockwise rotation disk drive, every rotational acceleration seekoperation to the first disk drive will be accompanied by an equal andopposite rotational acceleration seek operation to the second diskdrive. By forcing these rotational accelerations to be simultaneous,some or all of the rotational acceleration will be counteracted orcancelled. In some embodiments, the rotational acceleration due toactuator seek operations is minimized by sending simultaneous seekcommands to each drive of a coupled pair 430. This reduces error ratesand increases system performance. Further, because two drives areproviding the data, some aspects of data-transfer bandwidth can bedoubled. In some embodiments, the axis of rotational mass 1440 of eachdisk drive 1120 within a coupled pair 1430 is aligned to be collinear(lying on or passing through the same straight line or having axes lyingend to end along a straight line) with the axis of rotational mass ofthe other disk drive 120 of that coupled pair 1430. In some embodiments,one or more disk drives 1432 is not a member of a coupled pair. Forexample, if an odd number of operating drives is provided, or if one ormore drives fails, it is sometimes not possible for all drives to bemembers of respective coupled pairs. In some embodiments, spare drivesare provided in coupled pairs such that if one drive of one of theoperating sets of coupled pairs fails, the spare pair can be substitutedfor the coupled pair having the failed drive. In some embodiments, at alater time, it may be desirable to use the now-single remainingoperational drive of the swapped-out pair to be used in some capacity,(e.g., if all the spare pairs are used up, a single drive failure couldcause swap of the now-single remaining operational drive for the newlyfailed drive). Other aspects of system 1400 are as described for FIG.11, FIG. 12, and/or FIG. 13.

FIG. 15 is a plan view block diagram of a disk-drive-connector circuitcard system 1500 used in some embodiments of the invention. In someembodiments, a rear circuit card 1512 has a relatively short centeraspect such that its connector 1513 is closer to the top of circuit cardsystem 1500 (as illustrated in FIG. 15) and circuit card 1514 has arelatively longer center aspect such that its connector 1515 is alsocloser to the top of circuit card system 1500 (as illustrated in FIG.15). By having connector 1515 closer to the top of the circuit cardsystem 1500, a shorter perpendicular connector card can be used toconnect connectors 1515 and 1513 to the top of the circuit card system1500 (in some embodiments, this is the back of the enclosure whichincludes the air outlet side of the enclosure). In some embodiments, gap1520, between circuit card 1514 and circuit card 1512, matches the angleof the space between disk-drive connectors at its edges 1522, 1524,1526, and 1528. Thus, gap 1528 is at an angle to the front of theenclosure that matches the angle of the disk drives in that respectiverow. Accordingly, the gap 1528 between circuit cards 1512 and 1514 fallsmidway between two neighboring disk-drive connectors. Thus thecontinuity of disk drive spacing within a row is not interrupted. Thisallows the connectors that are adjacent to these edges to be completelyon either circuit card 1512 in the case of a connector on one side ofgap 1520 or on circuit card 1514 in the case of the connector adjacentthe other side of gap 1520. The pattern of gap 1520 further allows an atleast approximately equal number of disk drives to be placed on circuitcard 1512 as are placed on circuit card 1514 while still mountingconnector 1515 closer to the top of circuit card system 1500. Bysplitting the connector circuit onto two cards, the longest dimension ofeach card is reduced, making manufacturing easier and less expensive,and increasing yields. In some embodiments, the length of circuit cardsystem 1500 is approximately 32 inches (about 81.28 cm), and the widthis about 17 inches (about 43.18 cm), however each of the cards 1512 and1514 have a length and width each less than about 20 inches (50.8 cm)making fabrication easier and less costly than if longer dimensions areused.

In some embodiments, redundant power supplies are provided for eachcircuit-card portion. For example, in some embodiments, two DC-to-DCpower supplies, either of which could alone supply sufficient power forabout fifty disk drives, are provided and connected to one side of eachcircuit board 1512 and 1514 (e.g., the bottom side, in someembodiments), and about fifty disk-drive sockets are provided andconnected to the opposite side of each circuit board 1512 and 1514(e.g., the top side, in some embodiments). In some embodiments, DC-to-DCpower supplies that use forty-eight volts input, and that supply one ormore output voltage and current values, as required by the disk drives,are used. In other embodiments, three such DC-to-DC power supplies, anytwo of which could supply sufficient power for about fifty disk drives,are provided for each circuit-board portion 1512 and 1514. In stillother embodiments, other power-supply configurations are used.High-reliability relays of the type used in automotive applications andhaving almost no internal voltage drop across the relay contacts (unlikesolid-state relays which typically dissipate a not-insubstantial amountof power) are used, in some embodiments, to selectively connect thepower supplies to the disk drives when desired or disconnect them if afailure is detected. In some embodiments, these relays are used tosequentially connect a few drives at a time upon power-up, in order toreduce the power surge due to spin-up of the disks.

FIG. 16A is a plan view block diagram of a data-storage system 1600 ofsome embodiments of the invention that provides a high-density enclosurehaving (in this exemplary embodiment) four rows of disk drives. In someembodiments, system 1600 is mounted (e.g., in a rack) with its majorfaces horizontal, the front side with air inlet ports 1201 being at thebottom of FIG. 16A, the back side with air outlet ports 1202 at the topof FIG. 16A, and with the left and right sides of enclosure 1610 beingclosed. Inlet air 1113 is guided toward the back or top of inletmanifold 1212, and a little of this air splits off between each pair ofdrives 120 to cool the disk drives, and the warmed outlet air 1115collects in outlet manifold 1214 and is drawn by fans 1615 through andout outlet ports 1202. In some embodiments, the disk drives themselvesact as heat-sink fins (e.g., of the enclosure as a whole, as well as forthe electronic circuits on circuit boards 1512 and 1514 and the diskdrives themselves), both directing air flow and conducting heat into theair flow passing though the spaces between the disk drives. Referencenumbers in FIG. 16 that are not explicitly described here refer toelements discussed previously and shown in earlier Figures.

Each row 1650 has a plurality of drives (in some embodiments, up tofifty or more disk drives 120). Front wedges 1618 provide air passagesin front of the front-most disk drives, and back wedges 1616 provide thesame function for the rear-most disk drives, thus assuring that each andevery disk drive receives the appropriate amount of air flow on bothsides of every drive. In some embodiments, blank spacers are placed atsocket positions that do not have disk drives in order that air flow isnot disrupted by blank openings where disk drives are missing (air flowgoing through the path of least resistance).

In some embodiments, a centrally mounted personality board 1660 isplugged into socket or connector 1515 of the front circuit board 1514and into socket or connector 1513 of the rear circuit board 1512.

FIG. 16B is a functional block diagram of a circuit 1608 used in someembodiments of system 1600. In some embodiments, a plurality of Mfirst-level fanout-fanin expander circuits 1664 are each connected to aplurality of disk drives 120 (e.g., each circuit provides N=six, eight,ten, twelve, or some other number of “downward busses” 1668 to a likenumber of disk drives) and each fanout-fanin expander circuit 1664provides one or two intermediate “upward busses” including upward bus1666 onto which is placed the consolidated data traffic to and from theN drives (e.g., a first upward bus onto which system data is sent orreceived, and a second upward bus that remains in the enclosure forstatus, data reconstruction, and display purposes). In some embodiments,the second upward bus 1663 from each of the first-level fanout-fanincircuits are fed directly, or through further fanout-fanin concentratorcircuits that feed into, a status controller or maintenance computer1669 in the enclosure, which tracks status of all the drives, and if adrive has failed or has been detected to be in a condition thatindicates the drive is about to fail, the data from that drive isreconstructed (for example, the data is copied from a drive that mirrorsthe data on the failed drive) and placed on a spare drive, that fromthen on is used in place of the failed drive. In some embodiments,status controller 1669 also provides a driver to display variousmessages on display 930 as described for FIG. 9A, FIG. 9B, and FIG. 10.

In some embodiments, the first M upward busses 1666 are in turnconsolidated through further fanout-fanin expander circuits 1665 to afewer K number of upward externally-presented data busses 1661. In someembodiments, personality board 1660 includes electronic circuits thatprovide some or all of the circuitry for presenting upwardly a pluralityof serial attached SCSI (SAS) busses (e.g., about ten to abouttwenty-five busses, in some embodiments, providing connectivity to abouttwo-hundred disk drives), or alternatively, provide a plurality ofserial ATA busses (e.g., about ten to about twenty-five busses, in someembodiments).

As shown in FIG. 16C, in some embodiments, each of these serial externalbusses 1661 is connected to its own electronic fanout-fanin circuit 1672that connects either directly to a plurality of disk drives (e.g., oneor two external busses on one side of the circuit and eight, ten, ortwelve disk drives each connected to its bus 1666 on the other side ofcircuit 1672, in some embodiments), or connects to further levels offanout-fanin circuitry as shown in FIG. 16B.

Referring again to FIG. 16A, in some embodiments, the split line (thedemarcation) 1520 between the plurality of boards is made such that allconnectors for disk drives 120 or connectors for personality board 1660are completely on one board (e.g., 1512) or another (e.g., 1514). Insome embodiments, a first plurality of on-board DC-to-DC power supplies(e.g., three power supplies, in some embodiments) is connected to board1512 and selectively switched to provide redundant power to theplurality of disk drives that are connected to board 1512, and a secondplurality of on-board DC-to-DC power supplies (e.g., three powersupplies, in some embodiments) is connected to board 1514 andselectively switched to provide redundant power to the plurality of diskdrives that are connected to board 1514. In some embodiments, a set ofsequentially activated switches (e.g., solenoid-controlled relays) oneach board are connected from the various power supplies to differentsubgroups of disk-drive connectors, in order to reduce the magnitude ofsurge current that is drawn by the disk drives as they spin-up.

FIG. 17 is a plan view block diagram of a data-storage system 1700 ofsome embodiments of the invention that provides a high density enclosurehaving one or more rows of disk drives accommodating a variable numberof disk drives in each row. In some embodiments, one or morevariable-width air-flow blocks (or spacers) 1771 and 1772 are providedto fill the space or spaces that are not currently occupied by diskdrives. In the embodiment shown, two disk drives 120 and 1120′ areinitially provided, and variable-width air-flow blocks 1771 and 1772(e.g., each including a stack of disk-drive-width spacers, equal innumber to the missing disk drives) are provided to fill all the otherdisk-drive spaces. In other embodiments, the disk drives are inserted atthe front or back end of the row, and a single spacer (e.g., 1771) isused. As additional drives are inserted into system 1700, the widths ofair-flow blocks 1771 and/or 1772 are decreased. In this way, row 1650 isable to accommodate a variable number of disk drives and maintainappropriate air flow around all of those disk drives that are provided.Reference numbers in FIG. 17 that are not explicitly described hererefer to elements discussed previously and shown in earlier Figures. Insome embodiments, the shape of row 1750 is straight and oriented at aright angle to air inlet side 1718, similar to row 1150 as shown in FIG.11. In some embodiments, the variable-width air blocks 1771 and 1772 areadjustable in different increments, to accommodate the varying spacingsbetween disk drives from the front to the back of the row. In someembodiments, the shape of row 1750 is straight and oriented at an acuteor diagonal angle to the air inlet side, similar to row 1250 as shown inFIG. 12. In some embodiments, the shape of row 1750 is curved and at anacute angle to air inlet side 1718, similar to row 1350 as shown in FIG.13. In some embodiments, adjacent pairs of disk drives in row 1750 isstaggered, similar to row 1450 as shown in FIG. 14, as well as beingcurved (e.g., as in FIG. 13) and/or at a diagonal angle (e.g., as inFIG. 12) and/or at a right angle (e.g., as in FIG. 11). In someembodiments, two or more such rows (either as shown or mirror image, oralternating as shown and mirror image—e.g., as in FIG. 16) are arrangedside-by-side in a single enclosure.

In some embodiments, the number of functionally utilized disk drives isfewer than the number that could be placed in an enclosure (e.g.,one-hundred-seventy-two of a possible one-hundred-ninety-two, in someembodiments (e.g., four rows of forty-eight drives per row)) and avariable number of spare drives are provided (e.g., up to twenty sparedrives, in some embodiments), wherein the number of spare drivesprovided is variable and set by calculating the number needed to providea given system lifetime to a given probability (e.g., ninety-eightpercent probability of lasting three years without running out ofspares, or 99.9 percent probability of lasting five years withoutrunning out of spares). Given a predicted failure-rate curve for theentire population of disk drives, the number of disk drives to be usedfunctionally, and perhaps other parameters such as the expectedtemperature inside the enclosure over time), the number of spare drivesneeded is calculated. In other embodiments, the total number of drivesis fixed (e.g., one-hundred-ninety-two disk drives), and the number ofdisk drives to be used functionally (and thus the total data capacity)is varied, such that the other drives provide sufficient spares for theexpected lifetime of the enclosure.

In some embodiments, the enclosure is delivered to the customer with astated total capacity (based on the number of disk drives to be usedfunctionally, e.g., one-hundred-seventy-two), and with a given number ofspare drives (e.g., twenty). Over time, individual ones of thefunctional disk drives will fail and be replaced using the spare drives.In some embodiments, the data on each drive is mirrored on acorresponding disk drive of an adjacent row, and the data space isstriped over a row of drives. In some embodiments, if one drive of sucha mirrored pair fails, its data is reconstructed to both drives of aspare pair of drives using data from the mirror drive of the faileddrive, and the spare pair will thereafter be used in place of the pairwith the one failed drive. For example, the above enclosure could beconfigured as eighty-six pairs of functional drives and ten pairs ofspares (i.e., totaling one-hundred-ninety-two disk drives). During aninitial “pair-of-drives-swap” phase, if either drive of a pair fails, aspare pair is loaded with recovered data from the remaining good drive,and that spare drive is swapped for the pair having one failed drive.Later, once all the paired spares have been used to replace pairs ofdisk drives (each pair having only a single drive that has failed andanother drive that is still good), a second “single-drive swap” phase isused, wherein when a single-drive failure is detected, its recovereddata is placed on the remaining single good drive of one of the pairsthat was swapped out. In some embodiments, during the initial“pair-of-drives-swap” phase, the reduced rotational vibration (RAV)characteristic is maintained by swapping a pair of drives having reducedRAV (e.g., counter-rotating drives or drives at orientations, e.g., atright angles, that reduce RAV effects) for a pair having a failed drive,and during the later “single-drive swap” phase, the slight loss orreduction in RAV resistance is tolerated or compensated for by somewhatreduced performance.

FIG. 18 is a perspective view block diagram of a data-storage system1800 of some embodiments of the invention that provides one or more rows1750 of disk drives 120 in an upper portion of the enclosure and one ormore power supplies in an adjacent lower portion of the enclosure. Insome embodiments, one or more rows 1750 of disk drives are connected tothe top side of connector plate or circuit board 1500. In someembodiments, each disk drive is a 0.35-inch (9 mm) thick, 2.5-inch (6.35cm) form-factor unit that is plugged into a corresponding socket (e.g.,either parallel ATA (PATA), serial ATA (SATA) or serial SCSI (SSCSI))that is soldered to the upper surface of connector board 1500. In someembodiments, one or more power supplies 1866 are connected to the lowersurface of connector board 1500. Reference numbers in FIG. 18 that arenot explicitly described here refer to elements discussed previously andshown in earlier Figures.

FIG. 19 is a cutaway side view of a data-storage system 1900 of someembodiments of the invention that provides a high density enclosurehaving one or more rows of disk drives. Reference numbers in FIG. 19that are not explicitly described here refer to elements discussedpreviously and shown in earlier Figures. In some embodiments, a firstface 1121 of each disk drive (e.g., 120 and 1120′) is facing onedirection (left in the figure) along the axis of row 1150 and theopposing second face 1122 is facing the opposite direction (right in thefigure) along row 1150. For example, in some embodiments, for each diskdrive, the first face 1121 includes a metal cover 1123 that covers thedisks and actuator and opposite side (second face 1122) includes aprinted circuit card 1124 that holds the electronics for the disk drive120. Along one side (e.g., the side facing the viewer in FIG. 19) ofdisk-drive row 1150 is an air-inlet manifold that conveys inlet air 1113to the closer edge of the disk-drives 120 in row 1150. Along one side(e.g., the side facing the viewer in FIG. 19) of disk-drive row 1150 isair-inlet manifold 1112 that conveys inlet air 1113 to one edge of thedisk-drives 120 in row 1150. The bottom edge (as viewed in FIG. 19) ofeach disk drive 120 has a connector (e.g., two rows of pins) thatconnects to connector 1923 that is mounted to connector board 1500(e.g., in some embodiments, connector 1923 is a socket configured toreceive the pins of the disk-drive connector). In some embodiments, aresilient (e.g., elastomeric or visco-elastic) boot (or other shape thatconnects disk drive 120 to connector board 1500 and/or to connector1923) 1972 provides a mechanical connection between each disk drive 120and connector board 1500 that absorbs vibrations (such as fromactuator-caused rotational acceleration or vibration) that otherwisewould be transmitted from one disk drive 120 to another 1120′. In someembodiments, at the opposite side (e.g., the top of each drive in FIG.19), an adhesively connected resilient (e.g., elastomeric orvisco-elastic) disk-drive-cap material 1971 connects the side or edgeopposite the connector edge of each disk drive 120 to top cover 1970(e.g., a plate of sheet steel or aluminum or reinforced composite).Disk-drive-cap material 1971 provides mechanical support for each diskdrive 120 by providing a double-sided adhesive structure that, togetherwith connector 1923 and/or boot 1972, holds the disk drive in place. Insome embodiments, disk-drive-cap material 1971 provides avibration-dampening function (e.g., absorbing vibration energy andconverting it to heat). In some embodiments, no screws, shuttles, orother mechanical structures are used to hold drives 120. This allowsmore moving air 1111 to contact and cool the disk drives 120, reducesweight of data-storage system 1900, and simplifies and reduces the costof assembly. In some embodiments, one or more power supplies 1866 havepins 1867 that are soldered to through holes in connector board 1500 andpower supplies 1866 are thus attached to the bottom side of board 1500opposite the disk drives. In some embodiments, bottom cover 1979 (e.g.,a plate of sheet steel or aluminum or reinforced composite) is placed incontact with a surface of power supply 1866 to provide a heatspreader/heat sink, and air passes around the lateral sides of powersupply 1866.

FIG. 20A is an elevation view of a data-storage system 2000 of someembodiments of the invention that provides a high density enclosurehaving one or more rows of disk drives arranged in coupled pairs ofcounter-rotating disk drives. In some embodiments, data-storage system2000 is similar to system 1900 of FIG. 19, except that at least some ofthe disk drives 120 are placed back-to-back (within a pair) andfront-to-front between pairs. Further, in some embodiments, a perforatedplate 2073 (e.g., a plate of sheet steel or aluminum or reinforcedcomposite) is provided (in some embodiments, in place of the boots 1972shown in FIG. 19, or, in other embodiments, in addition to boots 1972)such that an opening is provided for each of a plurality of disk drives120, and a resilient (e.g., elastomeric or visco-elastic) material 2074bridges at least some of the gaps between the disk drives 120 and plate2073. In some embodiments, resilient material 2074 is much larger inheight and width than is shown in FIG. 20A, and provides significantdampening of vibrations of drives 120. In some embodiments, the heightof plate 2073 with respect to the connector edge of the disk drives isvariable, in order to be able to select a position that best dampensvibrations. In some embodiments (for example, movable by screwadjustment to different distances from connector board 1500), the heightof plate 2073 with respect to the connector edge of the disk drives isdifferent for various drives in a single enclosure, in order to be ableto select a configuration that best dampens vibrations. In someembodiments, a plurality of fans 1614 and 1615 (optionally in differingvertical positions) are provided to urge air flow through the enclosure,both around and/or in between (e.g., flow 1111) disk drives and aroundand/or between power supplies 1866. In the embodiment shown, fan 1614provides both flows 1813 and flows 1111, while fan 1615 provides mainlyflows 1111. In some embodiments, a plurality of other disk drives 120are faced in alternating directions to the left and right of the diskdrives 120 shown here, in order to help cancel or reduce rotationalaccelerations transmitted between disk drives 120.

In some embodiments, a large plurality of disk drives (e.g., in someembodiments, the number of drives equals 48, 50, 96, 100, 150, 172, 192,200, or more disk drives, and four, six or more power supplies) areadhesively held in the enclosure of system 2000, with a sufficientnumber of spare drives (e.g., ten, 16, 20, or more spare disk drives)such that the enclosure can be placed in service with the expectationand probability that enough spares have been provided to allow thesystem to remain in service for the expected lifetime (e.g., three yearsor five years or other selected periods) without needing a field-servicecall.

FIG. 20B is an elevation view of a data-storage system 2001 of someembodiments of the invention that provides a high density enclosurehaving one or more rows of disk drives with an adjustable-heightmid-drive vibration damper 2075. In some embodiments, mid-drive damper2075 is made of or includes a visco-elastic material, elastomericmaterial, resilient material, or the like. In some embodiments, a metalgrid such as grid 2073 of FIG. 20A is embedded in, placed under, orotherwise supports damper 2075. In some embodiments, at the oppositeside from the electrical connector (e.g., the top of each drive in FIG.20B), an adhesively connected resilient (e.g., elastomeric orvisco-elastic) disk-drive-cap material 1971 connects the side or edgeopposite the connector edge of each disk drive 120 to top cover 1970(e.g., a plate of sheet steel or aluminum or reinforced composite or thelike). Disk-drive-cap 1971 and mid-drive damper 2075 provide mechanicalsupport for each disk drive 120 by providing adhesive structures that,together with connector 1923 (and/or boot 1976 shown in FIG. 20C), holdthe disk drive in place. In some embodiments, disk-drive-cap 1971 andmid-drive damper 2075 provide a vibration-dampening function (e.g.,absorbing vibration energy and converting it to heat). In someembodiments, disk-drive-cap 1971 is omitted, leaving the mid-drivedamper 2075 to provide the support and vibration-absorption functions.In some embodiments, no screws, shuttles, or other mechanical structuresare used to hold disk drives 120, but rather disk-drive-cap 1971 andmid-drive damper 2075 together with connector 1923 provide the onlysupport and fastening for disk drives 120.

FIG. 20C is an elevation view of a data-storage system 2002 of someembodiments of the invention that provides a high density enclosurehaving one or more rows of disk drives with a cast-in-placevibration-damper boot 2076. In some embodiments, the disk drives 120 areinserted into their respective socket (or other electrical connector)1923, and a liquid or flowable dampening material is poured, injected,or otherwise placed around the base of each disk drive 120, andsolidified (e.g., by thermal, chemical, photonic, or other means) toform vibration-damper boot 2076. In some embodiments, one or moreopenings 2080 are provided in connector circuit board 1500 that allowthe visco-elastic material to flow between board 1500 and power supply1866 to provide additional dampening properties.

FIG. 20D is an elevation view of a data-storage system 2003 of someembodiments of the invention that provides a high density enclosurehaving one or more rows of disk drives with a cast-in-place mid-drivevibration damper 2077. In some embodiments, a mold 2078 (such as a sheetof stretchy plastic film having a slit or other suitable opening to fitover each disk drive 120) is placed over a plurality of the disk drives120, and a liquid or flowable vibration dampening material is poured,injected, or otherwise placed onto mold 2078 around a selected mid-point2081 each disk drive 120, and solidified (e.g., by thermal, chemical,photonic, or other means) to form mid-drive vibration-damper 2077. Insome embodiments, the mid-point location 2081, at which mid-drive damper2077 is placed, is not half-way between connector end 2082 and oppositeend 2083 of each drive, but is at some height around each disk drivethat is selected to improve vibration dampening. In some embodiments,the height is selected to be at approximately the center of rotationalvibration mass of each disk drive 120. In some embodiments, mold 2078 isa stretchy tube that is placed around the disk drives 120 and which isfilled with a material (such as a gas, liquid, or a material thatsolidifies) in order to stretch the tube into contact with drives 120.In some such embodiments, the tube is made of an adhesive-coatedresilient (e.g., elastomeric or visco-elastic) plastic material.

In some embodiments, two or more of the vibration dampening structuressuch as boot damper 1972, cap damper 1971, mid-drive damper 2073 and2074, mid-drive damper 2075, boot damper 2076, and/or mid-drive damper2077 are used in a single enclosure to combine to provide improveddampening.

FIG. 21 is a front elevation view of a data-storage system 2100 of someembodiments of the invention that provides a high density enclosurehaving one or more rows of disk drives 120 with one or more verticalbeam stiffeners 2110 and optional vibration damper 2122. System 2100includes an enclosure 2101 having side walls 2115, bottom plate 1979 andtop cover 961. In some embodiments, one or more side walls 2115 and/orcovers 1979 and 961 are at least partially coated (e.g., on their insidesurfaces) with a visco-elastic vibration-dampening sheet 2121, 2120 and2123, respectively. In some embodiments, visco-elasticvibration-dampening sheet 2121, 2120 and 2123 are attached on theinside, and in other embodiments, they are on the outside. In someembodiments, one or more side walls 2115 and/or covers 1979 and 961 areat least partially coated (e.g., on their outside surfaces) with an ESDcoating 2116 to dissipate static electric charge. In some embodiments,vertical beam stiffeners 2110 are attached to connector circuit board1500 and/or drive cap plate 1972 using elastomeric or visco-elasticmaterial 2111. In some embodiments, visco-elastic vibration-dampeningsheet 2120 is also adhesively attached (e.g., across most or all oftheir bottom surfaces) to power supplies 1866. In some embodiments,connector circuit board 1500 is held in place using elastomeric orvisco-elastic material 2112. In some embodiments, controller card 960 isattached (e.g., by a plug-and-socket 965) to center circuit board 966.In some embodiments, elastomeric or visco-elastic material 1971 isadhesively attached to the top of each disk drive 120 to hold it inplace (rather than using metal or plastic shuttles or other holdingdevices. In some embodiments, a stiffening ridge 2172 is welded to orattached using elastomeric or visco-elastic adhesive material to capplate 1972 and/or bottom plate 1979. In some embodiments (not shown) asimilar stiffening ridge is added to top cover 961.

Read-Splitting

Read-splitting is an important and valuable technique for increasing theperformance of disk arrays that use pairing. In some embodiments, datais “mirrored” to two or more drives (in a set of M drives, where M istwo or greater, each data write from the system causes the same data tobe replicated and written to each of the M drives). The data can bestriped as well (for example, eight drives can be configured asmirror-four and stripe-two, such that each write operation is replicatedfour times, and if the block spans more than one drive, each of the foursets of data is striped across two drives; alternatively, the data couldbe mirrored-two and striped-four, where the data is replicated twice,and long pieces of data are striped across four drives).

In some embodiments, when reading, every Mth read operation goes to thefirst drive of a set, every M+1^(st) read goes to the second drive, etc.This reduces the utilization of each drive, since only 1/M of the readsare directed to each drive. In other embodiments, each read operation issent to all drives, and the first drive to return data has its dataused, and the other drive's data is ignored or discarded. This increasesthe speed of retrieval, since the fastest drive provides the data.

Embodiment C1: Read-splitting Using Vibration-interaction Mapping (e.g.Wherein Physical Location of Drives Determines Which Drive is Used for aParticular Read Operation)

When data is read from a plurality of mirrored or striped/mirroreddrives using read-splitting logic in the RAID controller or software, itcan be highly unlikely that an I/O request to logical disk will causesimultaneous actuator movements among mirrored physical disk drives, andit is problematic to try to predict the direction and duration of theseaccelerations with respect to nearby disk drives. Rather, read-splittingand bus/loop arbitration logic among the disk drives makes it likelythat these accelerations will be random with respect to other drives,and therefore also likely that RAV energy created by one disk of amirrored set that is seeking will be transmitted to a nearby disk (the“subject” disk) that is in the process of transferring data to/from themedia, making the subject disk particularly vulnerable to RAV.

In some embodiments, a coordinated logical to physical mapping ofmirrored disk drives via RAID ensures that mirrored HDA's are orientedorthogonally (Embodiments B1, B2, and the like) to one another, whilestriped HDAs are oriented with alternating rotational directions(Embodiments A1, A2, and the like).

In some embodiments, a first data structure is kept (e.g., in theenclosure's controller-card memory) that maps the physical location (seeTable 1B below) and/or drive-to-drive vulnerability (see Table 1A below)of each drive of each mirrored set, and a second data structure is keptthat indicates the state (e.g., idle, seeking, reading, or writing,etc., and/or the actuator location or address of last data blockaccessed) of each drive. In some such embodiments, read splitting isused, wherein the determination of which drive of a mirrored set isselected to use for a given read-split read operation is based, at leastin part, on the state of nearby drives that could be affected by sendinga seek operation to a given drive. For example, if a read command isreceived by the enclosure's controller card that could be satisfied bysending the command to any one of a plurality of drives in a mirroredset, for each drive that can provide the requested data the controllerexamines the state (as specified by the second data structure) of thenearby or most vulnerable drives (as specified by the first datastructure), and the controller then selects the drive that is leastlikely to cause an error in its neighboring drives.

In some embodiments, the content of the first data structure, for eachdrive in the enclosure, specifies which other drives are most vulnerableto an RAV error due to a seek operation, and optionally specifies themagnitude of vulnerability (the probability of an induced RAV error). Insome embodiments, the content of the first data structure is determined,at least in part, by the physical location and/or orientation of eachdrive. In some embodiments, the content of the first data structure isdetermined, at least in part, by an empirical measurement, for eachdrive, of the drive-to-drive vulnerability as measured by establishing aread-tracking mode in a subject drive and then performing a seekoperation of a given magnitude in the drive being tested. For example,when determining the neighboring-drive-vulnerability mapping of thefirst drive (the seek drive), one at a time each one of the neighboringdrives (the victim drive) is forced into a state of read tracking, thefirst drive is then made to perform a seek operation, and it isdetermined whether the victim drive suffered a tracking error as aresult of the seek. In some embodiments, this is repeated a number oftime to ascertain the probability of a tracking error being caused. Insome embodiments, a plurality of different seek amounts or magnitudes(e.g., small, medium, or large) is tried for each seek drive during thedata structure generation, and the resulting tracking-errorprobabilities are determined for each of the other drives in theenclosure.

TABLE 1A Data Structure 1 Large seek or small seek or RAV large rotationacceleration vibration (RAV) medium seek or RAV . . . . error Vic- errorerror error Victim error Victim error Victim prob- tim prob- Victimprob- Victim prob- Seek Drive drive probability drive probability driveability . . . drive ability drive ability . . . drive ability . . . . .. 1 2 .6 5 .5 17 .15 . . . 2 .4 5 .1 . . . 2 .05 . . . . . . 2 1 .55 4.4 17 .3 . . . 1 .3 4 .2 . . . 1 .03 . . . . . . 3 2 .24 44 .24 1 .08 .. . 2 .14 4 .14 . . . 2 .024 . . . . . . . . . . . . . . . . . . . . . N147 .1 145 .09 12 .07 . . . 147 .1 145 .07 . . . 147 .02 . . . . . .

In use, suppose a read-split read is received and can be serviced byeither drive 1 or drive 3 (since the requested data isreplicated/mirrored on these two drives). If all the drives in row 1 androw 3 of data structure 1 are idle, then the enclosure controller cansend this read operation to whichever drive (1 or 3) would have theshortest seek or the least rotation acceleration, or a random choice orping-pong (i.e., alternating successive reads between these two datasources) choice between drive 1 and 3 could be made. Suppose, however,that drive 5 is in a read-tracking state: the entries for drive 1 showthat there is a non-negligible error probability (0.50) that a trackingerror will occur if the specified seek (suppose a large seek for thisexample), while the entries for drive 3 do not indicate that an error isprobable for drive 5 if a seek is performed on drive 3. Accordingly, theread-split read command will be directed to drive 3, since there islittle or no likelihood that a tracking error would result. Note alsothat, in some embodiments, data structure 2 provides actuator-locationinformation for each candidate drive, which when compared to the addressof the incoming read-split read command, provides the indication of thesize of the seek operation (i.e., the magnitude of the accelerationvibration that will be generated). In some embodiments, data structure 2provides a parameter for each drive of the enclosure's relativeflexibility or stiffness at that drive's location (and/or thenode-antinode parameter that indicates how close to or far from astanding-wave-resonance node that drive is positioned). In someembodiments, this stiffness and/or node parameter is an input into theformula used to determine the size of the seek or rotationalacceleration vibration that is used as an input to Table 1A (i.e., if adrive is positioned at a stiff location or near a resonance node, a seekthat would cause a “Large” RAV on another drive might cause only amedium or small RAV for this drive).

TABLE 1B Data Structure 1 drive—drive drive—drive drive—drive Victimspacing, Victim spacing, Victim spacing, Seek Drive drive orientationdrive orientation drive orientation . . . 1 2 6.0 cm, parallel 5 5.0 cm,17 1.5 cm, . . . 60 degrees in-line 2 1 6.0 cm. parallel 4 4.0 cm, 173.0 cm, . . . parallel parallel 3 2 2.4 cm, 44 2.4 cm, 1 0.8 cm, . . .orthogonal 20 degrees orthogonal . . . . . . N 147 1 cm, 145 19 cm, 1217 cm, . . . facing 25 degrees parallel

Rather than or in addition to Table 1A that tracks the drive-drivevulnerability to errors, in some embodiments, a data structure such asTable 1B is kept that stores the distance and/or relative orientationand/or node-antinode positioning and/or relative stiffness between aplurality of pairs of drives. Table 1B is used in a manner similar tothe use of Table 1A, in that an incoming read-split read operation isreceived by the enclosure controller, which then makes a choice betweenthe possible source drives for the requested data based on the distance,orientation, node-antinode positioning, and/or relative stiffnessbetween the selected drive and other drives that are in a state thatmakes them vulnerable to RAV-induced tracking errors.

TABLE 2 Data Structure 2 Enclosure relative flexibility Actuatorlocation or or stiffness at drive Drive Drive state sector last accessedlocation, or node-antinode 1 Idle track 3047 5 2 Idle track 1540 8 3Seek track 10 1 4 write tracking track 30205 7 5 read tracking track1540 6 . . . N Idle track 4222 2

Cabinet Design

One problem addressed by this present invention is created whenrotational vibration (movement that revolves around the axis of theactuator motor), usually from another drive, rotates the drive relativeto the actuator, and thus pulls the head off the track it is readingfrom or writing to. With drives that are mounted vertically, one problemis that RAV that raises or lowers one corner 119 of the drive 120,and/or lowers or raises the opposite corner 121 in the other direction.

One aspect of some embodiments of the present invention includespositioning and orienting each drive to achieve a desired flow patternand volume of cooling air through the enclosure. Another aspect of someembodiments of the invention includes positioning drives in theenclosure with a spacing, orientation, and/or location so as to reduceor minimize drive-to-drive RAV induced tracking (or other) errors.Another aspect of some embodiments of the invention includes timingand/or synchronizing access commands that are sent to the drives in theenclosure so as to reduce or minimize drive-to-drive RAV inducedtracking (or other) errors.

In some embodiments, the disk drives are electrically connected to aconnector on the disk-drive-connector circuit board 1500, but are heldin place in the enclosure primarily using a visco-elastic material thatcontacts each disk drive at one or only a few locations to ensure thatthe disk drive remains connected to the connector, are allowed to moveslightly within the constraints of the visco-elastic holder, have theirvibrations dampened by the visco-elastic holder, and still have asubstantial surface area exposed to the air flow through the cabinet tocool the drives. By eliminating the metal or plastic shuttle and/orscrews that are typically used to hold a disk drive in place, asubstantial weight reduction is achieved.

In some embodiments, because of the large number of operational diskdrives, and the large number of spare disk drives that can be swapped inif there is a failure detected, the drives can be sealed in place, forexample, by adhering every disk drive to the visco-elastic holder, andadhering the visco-elastic holder to the enclosure. The large number ofoperational and spare drives also allows meaningful statistical analysisof the failure rate and a determination of where the unit is in the lifecycle of the enclosure. A typical expected life of the enclosure can betailored by adjusting the number of spare drives, for example, yieldinga unit having a three-year expected lifetime with more usableoperational data storage space (e.g., using fewer spare drives, e.g.,perhaps ten initial spare drives and one-hundred-ninety operationaldrives), or yielding a unit having a five-year expected lifetime withless usable operational data storage space (e.g., using more sparedrives, e.g., perhaps twenty-five initial spare drives andone-hundred-seventy-five operational drives).

Controller Design

Another aspect of some embodiments of the invention includes mirroringor replicating data on a plurality of drives (which improves reliabilityand/or performance) so that each read command can be directed to one ormore drives in order to shorten access time (if the same command is sentto two or more drives, the one that returns the data fastest is used,which improves performance), reduce the average drive utilization (thecommand is sent to fewer than all the drives that have the data, so thatthe other drives remain available to perform other operations, which canalso improve performance). In some embodiments, one of a plurality ofdrives containing the data is selected based at least in part on whethernearby or vulnerable drives would suffer errors as a result (e.g., basedon such parameters as what state each of the other drives is in (e.g.,read-tracking, seeking, or idle), the relative probability that a seekin a selected drive will cause an error in another drive, the distancebetween drives, the relative orientation, stiffness, node-antinodepositions, etc.).

Another aspect of some embodiments of the invention includes sendingsubstantially simultaneous and substantially the same size seek commandsto counter-rotating drives that are positioned relative to one anotherso that the rotational accelerations cancel, at least to some extent.The term “counter-rotating drives” means a set of drives configured suchthat for every drive that receives a given seek command that causes agiven rotational acceleration around an axis, there is another drivepositioned such that the same seek command will cause substantially thesame rotational around substantially the same axis but in the oppositerotational direction (thus canceling some or all of the RAV seen byother drives). Such a set of drives can have any even number of drivesin the set (2, 4, 6, etc.). Data can be mirrored and/or striped acrossthe set of drives in order to have many or all of the commands sent tothe set of counter-rotating drives provide the RAV-canceling function.

Striping Using Two or More Disk Drives to Send Opposing RotationalAccelerations; and/or Counter-rotating Pairs; Each Drive in a PairPhysically Facing the Other

In some embodiments, a plurality of the drives are placed inopposite-facing pairs (either front-to-front, or back-to-back). Thesystem stripes all writes and reads so 1/N (half the data if mirroredpairs of drives are used; N=2) of the data goes to each clockwise driveand 1/N (or half) of the data is sent to each counterclockwise drive (ina pair, both drives move their respective actuators the same directionand amount, either both “clockwise” or both “counterclockwise” for agiven access relative to their own top cover, but since they areface-to-face, simultaneous clockwise accelerations are in oppositedirections relative to an outside frame of reference). The system-levelsectors are N times (e.g., twice if N=2) as big as drive sectors. Insome embodiments, all seek accesses that move the actuator are sentsimultaneously to the CW and CCW drive of a pair so the rotationalmoments cancel within the pair. In some embodiments, the timing of theseek operations is synchronized to better cancel the rotationalacceleration. The pair is mounted rigidly or semi-rigidly to oneanother, but held with elastomer or visco-elastic to the case so therotational changes cancel within the pair and do not transfer to thecase. In some embodiments, the “simultaneous” pairs of seeks do notoccur at the same time, but both seek portions take place while theother drive is preparing to do its seek or has just finished its seekacceleration, but has not started to do the read or write portion. Thus,both seek portions take place during a first portion of the operationwhen the other drive is not trying to stay on track but is stillsettling to its desired track, and the data-access portions both takeplace in a second portion of the operation when the other drive is notseeking. During the first portion, both drives are in a less-vulnerablestate and can tolerate more RAV. During the second phase, both drivesare in the more-vulnerable read or write mode where they must stay ontrack to avoid losing performance or data, and neither is generating RAVto disturb the other.

FIG. 4A, for example, shows a pair of disk drives 120 and 120′ pluggedinto connectors on bottom membrane 150.

In Some Embodiments, Avoid In-Line Or L-Type Orthogonal Positioning; UseT-Type Orthogonal Positioning

Drives are most sensitive to rotational-acceleration vibration (RAV)moves. To correct for this, in some embodiments, the corner of one driveis placed at the rotational “center” of next drive. Some embodiments usea T-orientation. In some embodiments, drives are placed in pairs asdescribed above, and each pair is placed at an angle (e.g., a T-typeright angle) to an adjacent pair.

Orthogonal or Staggered Positioning—Herringbone But With RotationalMoments of Inertia of One Drive (i.e., the Corner) Positioned at theActuator Center or Rotational Center-of-Mass of Adjacent Drive

What starts as rotational torque around the Z_(R)-direction can be madeto be X- or Y-movement to drives that are at right angles, and even todrives that are at other intersecting angles. For example, positioning afirst corner of a first drive next to the center of rotational mass ofan adjacent second drive means that rotational torque that moves thefirst corner downward is downward movement at the center of rotationalmass of the second drive, and thus will move the entire second drivedown rather than rotating it. Further, the angled orientation(intersecting planes of the disk drives) provides additional stiffeningof the enclosure, particularly in embodiments where the bottom edge ofeach drive is held to the connector circuit board 1500 and the top edgeis adhesively held, for example, to a visco-elastic sheet that isadhesively held to a top plate (e.g., a sheet-metal enclosure cover).This arrangement also allows a large amount of the total exteriorsurface of each drive to be exposed to the flow of cooling air, and forthe disk drives themselves to serve as air vanes and/or heat-sink finsto direct the flow of the cooling air flow.

Add Walls and/or I-beams as Stiffeners Parallel to Major Face ofDrives-perpendicular to Dotted Lines that Connect Adjacent Corners ofDrives

Some embodiments add walls 2110 and/or ridges 2172 at right-angles tothe bottom and/or top enclosure surface (which act as vibrationalmembranes). These stiffeners reduce vibration transferred betweendrives. Stiffener walls can be added across what would otherwise beantinodes, discussed below. In some embodiments, visco-elastic dampeningmaterials 1971, 2120, 2121, and/or 2123 are applied to walls 2121, orenclosure surfaces 961, 1979, and/or 1972 to dampen vibrations andreduce noise.

Use the Array-controller Card as I-beam Stiffener Down Center of Case,Between Rows of Drives

In some embodiments, center controller card 966 (e.g., the card thatreceives commands from other units and passes the appropriate commandsto the drives, buffers data, and/or does RAID generation and correctionof data) can act as an alternative or additional stiffener to the wallsdiscussed above.

Include a Visco-elastic Dampener to Attach Face of Controller PC Boardto Face of Steel or Fiberglass I-beam

A visco-elastic adhesive or other such material attached across a wallface acts to dampen vibrations in that wall. In some embodiments,visco-elastic material with one or more adhesive faces is adhered tosuch walls and other enclosure covers, and/or is used to connect wallsto bottoms, covers, and intermediate structures to stop transmission ofvibrations from one structure to another. In some embodiments,visco-elastic material is adhered to circuit boards as well.

Include a Visco-elastic Dampener to Attach One or More I-beams to Topand/or Bottom Covers

A visco-elastic adhesive attached between walls at right angles to oneanother (e.g., to connect them to each other) acts to dampen vibrationsthat would otherwise transfer between those walls.

Place Drives at Node (Lesser-vibrating) Positions of the Standing-wavePattern of the Bottom Membrane of the Disk—Drive Array Enclosure

Conventional multi-drive disk storage subsystems place all the diskdrives adjacent an outer surface, typically each one with one of its twosmallest faces pointing outward along the front panel of the enclosure,with the opposite small face, having the connectors, plugged into anoutward-facing connector socket. This is needed in order to have accessto drives in case they need to be serviced or replaced in the field(e.g., by hot-unplugging the failed drive and hot-plugging the newreplacement drive in its place).

In contrast, some embodiments of the present invention include amuch-larger number of physically small drives mounted in their enclosurein a manner intended not to be replaced in the field, and with asufficient number of spare drives that can be logically swapped in placeto the number of drives that could be expected to fail during theservice life of the system. The operating drives are known as“fail-in-place” drives, since if and when they fail, the failed drive isleft physically in place in the enclosure, and one of the spare drivesis logically connected in its place and loaded with reconstructed dataof the failed drive.

Two-dimensional surfaces (membranes) have vibrational-resonance patternsthat are affected by the constrained edges (such as the outer edges ofthe bottom surface of the multi-drive enclosure disk storage system)

Some embodiments place drives in a pattern in the enclosure that matchesmore closely the non-vibrational node locations of the “membrane”surfaces (e.g., the bottom cover and/or wiring grid) to which they areattached. On a membrane, standing waves form a two-dimensional pattern,in which the constrained edges and other locations within the membranehave little or no standing-wave vibration, and other antinode locationshave much vibration. The node/antinode locations are affected by thesize, shape, and thickness of the membrane, as well as the other masses(e.g., disk drives and controller cards) and stiffeners (e.g.,right-angle walls and/or I-beams). In some embodiments, these node andantinode locations are determined empirically by placing the drives onthe surface, measuring the vibrational susceptibility and/or thenode/antinode pattern (i.e., whether drives in a particular locationsuffer seek errors, or the magnitude of vibration at each drive asdetermined by, e.g., vibrational holography, in which a photosensitivefilm is exposed to the interference pattern between a reference beam,and another beam that is split from the reference beam and illuminatesthe membrane surface while it is being acoustically stimulated to formstanding waves, as is well known in the art), then iteratively movingone drive slightly from its initial position and re-measuring until thatdrive reaches a point of minimum vibration; then iteratively repeatingthe process for neighboring drives until each drive is at a point on themembrane that is less or minimally RAV vulnerable (i.e., susceptible toread or write errors from received rotational acceleration vibration ofother drives), and/or minimally RAV dangerous (i.e., capable of causingrotational acceleration vibration that is transmitted to other drives).

FIG. 4A is a perspective drawing that illustrates a hard-disk-drive(HDD) or disk drive 120 mounted in a vertical orientation into connector126 integrated on a substrate printed circuit board (PCB) 150. The HDD120 has a side A 190, a side B 492, an edge C 194, an edge D 196, and aconnector 116 on the bottom edge. The HDD 120 has drive electroniccircuit board 150 attached to side B 492. Internal to the HDD 120 is aset of one or more disks 115, and an actuator assembly 112. The actuatorassembly 112 contains an R/W head 114. The actuator assembly 112 pivotsaround an axis of rotation 111 to perform seek operations.

When the actuator assembly 112 accelerates in one direction 191 oranother to perform a seek operation, there is a corresponding counterrotation force or torque 192 in the HDD 120, as a whole, producing arotational vibration. Since the mass of the HDD 120 is many timesgreater than mass of the actuator assembly 112, the magnitude of therotation of HDD 120 produced is much smaller than the magnitude of theactuator assembly rotation. This acceleration-induced torque 192rotating the HDD 120 produces rotational-acceleration vibration which istransferred to surrounding supporting structures such as the connector126 and substrate 120. The characteristic “click, click, click” that cansometimes be heard during actuator seek operations is due partly to therotational-acceleration vibration of the HDD 120.Rotational-acceleration vibration generated by drive 120 can causevibration in neighboring drive 120′ through supporting structures.Rotational-acceleration vibration is generally more problematic forcloser neighboring drives than those further away. Therotational-acceleration vibration interaction between hard-disk drives(HDDs) can cause actuator assembly seek or tracking problems inclose-neighboring drives. The present invention orients each of thedrives in an enclosure to reduce or minimize drive-to-drive coupling ofrotational vibration.

In some embodiments, the invention provides an apparatus that includes asubstrate, and a plurality of disk drives each coupled electrically andmechanically to the substrate, the plurality of disk drives including atleast a first and a second disk drive, wherein the first disk drive ispositioned relative to the second disk drive so that a rotational forceproduced by the first disk drive is at least partially counteracted by arotational force produced by the second disk drive.

In other embodiments, the apparatus can further comprise an enclosure,wherein the substrate and the plurality of disk drives are attached tothe enclosure, at least one memory, and an information processing unitoperatively coupled to the disk drives and to the memory, wherein theinformation processing unit sends read commands to the disk drives andreceives data from the disk drives and from the memory.

The apparatus can optionally include an information processing unit thatincludes a multi-processor supercomputer. In some embodiments, theapparatus includes a plurality of substantially similar enclosures,wherein each enclosure holds a substrate and plurality of disk drivesincluding at least a first disk drive and a second disk drive positionedsuch that a rotational force produced by the first disk drive isconveyed primarily as a translational force to the second disk drive,and wherein the plurality of enclosures are operatively coupled to thesupercomputer.

In some embodiments, the apparatus further comprises a memory and avideo-streaming apparatus operatively coupled to receive data from thememory, wherein the video-streaming apparatus is adapted to transmitdigital video to a plurality of destinations and users. In some otherembodiments, the plurality of disk drives includes more than two firstdisk drives in a first rotating orientation and fewer than aboutone-hundred-and-one first disk drives, and a substantially equal numberof second disk drives in a second counter rotating orientation, whereina plurality of the first and a plurality of the second disk drives areinterleaved in coupled pairs. In some embodiments, the plurality of diskdrives includes more than about one-hundred first disk drives and fewerthan about two-hundred-and-one first disk drives, and a substantiallyequal number of second disk drives, wherein a plurality of the first anda plurality of the second disk drives are interleaved in coupled pairs,each pair including one disk drive in a rotating orientation, andanother disk drive in a counter rotating orientation.

In some embodiments of the invention, at least some of the plurality ofdisk drives are each in contact with a boot unit. In some embodiments,the boot unit includes one or more resilient materials. In otherembodiments, the boot unit has graded shock absorbance characteristics.In still other embodiments, the boot unit includes a vibration dampingpolymer. A boot unit can include a visco-elastic material. In someembodiments of the invention, a first edge of each one of the pluralityof disk drives are adhesively connected to its boot unit. In otherembodiments, a first edge of each one of the plurality of disk drives isbonded to its boot unit.

In some embodiments, an apparatus of the invention can further include adetent device that is adapted to be placed in disengageable contact witheach one of the plurality of disk drives at an edge distal from thedrive's first edge. In some embodiments, the detent device is wedgeshaped at a first end and adapted to be inserted against each of aplurality of drives for transport and disengaged for disk operation. Inother embodiments, the detent device includes a cam mechanism adapted tobe engaged for transport and disengaged for disk operation.

In some embodiments of the invention, the first disk drive has a diskrotational torque vector due to its rotating disk(s) that issubstantially antiparallel to a disk rotational torque vector of thesecond disk drive that is due to its rotating disk(s). In someembodiments, the disk rotational torque vector of the first disk driveis substantially collinear with the disk rotational torque vector of thesecond disk drive. In other embodiments, the disk rotational torquevector of the first disk drive is radially offset from the diskrotational torque vector of the second disk drive. In still otherembodiments, the actuator rotational torque vector due to actuator armrotation in the first disk drive is substantially collinear with theactuator rotational torque vector of the second disk drive.

In some embodiments, a first major face of each of the first and seconddisk drive each have a first heat-conduction characteristic and thesecond opposing major face of the first and second disk drive have asecond heat-conduction characteristic that is different from the firstheat-conduction characteristic. In some embodiments, the first majorfaces of the first and second disk drives each are substantiallymetallic. In other embodiments, the first major faces of the first andsecond disk drives are each portions of a respective metal cover thatcovers the respective disk drive's disk(s) and actuator arm. In stillother embodiments, the second major faces of the first and second diskdrives each are substantially non-metallic. In some embodiments, thesecond major faces of the first and second disk drives each include aprinted circuit board. In some embodiments, the second major faces ofthe first and second disk drives each are substantially plastic, such asa fiberglass-reinforced epoxy circuit board.

In other embodiments, the first disk drive and the second disk drive arecoupled to the substrate with the first major face of the first diskdrive facing with a partial offset the first major face of the seconddisk drive. In still other embodiments, the first disk drive and thesecond disk drive are coupled to the substrate with the first major faceof the first disk drive facing with no offset the first major face ofthe second disk drive. In some embodiments, the first and second diskdrive form a first coupled pair, further including a second coupled pairhaving a third and fourth disk drive with a first major face of thethird disk drive facing with no offset a first major face of the fourthdisk drive, and a second major face of the second disk drive facing withpartial offset a second major face of the third disk drive.

In some embodiments, the apparatus can further include a controller thatreceives a disk access request specifying a data length of 2L and basedon the request sends substantially simultaneous disk access requests tothe first and second disk drive each specifying a data length of L. Insome embodiments, the substantially simultaneous disk access requestsent to the first and second disk drives cause seek operations havingrotational forces that at least partially cancel each other.

In some embodiments of the invention, the plurality of disk drives areformed into coupled pairs having substantially opposite rotationaltorque within each pair. In other embodiments, a first edge of eachcoupled pair is coupled to the substrate and an opposing second edge iscoupled to an elastomeric material.

In some embodiments, an apparatus can further comprise a stabilizermember having an elastomeric material in contact with at least some ofthe plurality of the disk drives between the first edge and the secondedge of the respective disk drives. In some embodiments, the stabilizermember is a plate member having an elastomeric material in contact withat least some of the plurality of the disk drives between the first edgeand the second edge of the respective disk drives. In some embodiments,the plate member is substantially parallel to the first and second edgeand includes a plate having perforations that encircle each disk drive.In other embodiments, the plate member further includes an elastomericmaterial bridging a gap between an edge of a perforation in the platemember and the disk drive encircled by the perforation.

In some embodiments, the plurality of first disk drives and second diskdrives are oriented as alternately facing coupled pairs. In otherembodiments, for each one of a plurality of disk drives, the first majorface of the respective drive is spaced closer to its nearest neighbor'sfirst major face as compared to the spacing of the respective drive'ssecond major face to its nearest neighbor's second major face, thesecond major faces having lower heat conductivity than the first majorfaces. In some embodiments, for each one of a plurality of disk drives,the first major face of the respective drive is spaced further from itsnearest neighbor's first major face as compared to the spacing of therespective drive's second major face to its nearest neighbor's secondmajor face, the second major faces having lower heat conductivity thanthe first major faces.

In some embodiments, the plurality of first and second disk drives areeach coupled electrically and mechanically to the substrate in a rowthat conforms to a line, wherein the first disk drives and the seconddisk drives are facing in alternate directions positioned within therow. In some embodiments, the row includes two or more disk drives andfewer than two-hundred-and-one disk drives. In other embodiments, eachof the first disk drives have a first major face and a second opposingmajor face and wherein each of the second disk drives have a first majorface and a second opposing major face, and wherein the first major faceof each first disk drive faces the first major face of an adjoiningsecond disk drive, and the second major face of each first disk drivefaces the second major face of an adjoining second disk drive. In someembodiments, the row conforms to a substantially linear line. In someembodiments, the row conforms to a substantially stepped curved line. Inother embodiments, the stepped curved line curves in a substantiallyexponential manner. In still other embodiments, the row conforms to asubstantially smooth curved line. In some embodiments, the substantiallysmooth curved line curves in a substantially exponential manner. In someembodiments, the apparatus includes one or more additional rows of diskdrives. In some embodiments, the rows are positioned on the substratewith substantially mirror image orientation relative to a neighboringrow.

In some embodiments, the apparatus further includes elastomeric materialthat is attached to the disk drives at a position on each of the diskdrives that is opposite the position on the disk drives proximal to thesubstrate.

In some embodiments, the apparatus further includes an enclosure. Insome embodiments, the substrate is oriented parallel to a first majorsurface of the enclosure. In some embodiments, the enclosure of theapparatus includes at least one air inlet and at least one air outlet.In some embodiments, the apparatus further includes at least onemanifold that directs airflow over the disk drives. In some embodiments,the apparatus further includes an air-movement-causing device. In someembodiments, the air-movement device includes one or more fans. In otherembodiments, the air-movement device includes one or more pairs of fansthat rotate in opposite directions. In some embodiments, the enclosureof the apparatus includes a cover. In other embodiments, the coverincludes a resilient material. In some embodiments, a resilient materialis attached to a second edge of each one of a plurality of the diskdrives. In some embodiments, the cover of the apparatus includes atleast one stiffening rib. In some embodiments, a resilient material isattached to the cover. In some embodiments, the apparatus furtherincludes a shipping-overshock display. In other embodiments, theapparatus further includes a mother board, a personality board, or anycombination thereof.

The invention provides a method that includes mounting a plurality ofdrives in an enclosure, the enclosure including a connector substrate,the plurality of drives including at least a first disk drive and asecond disk drive that are each electrically and mechanically coupled tothe enclosure, and mechanically coupling the first drive and the seconddrive such that rotational force produced by the first disk drive is atleast partially counteracted by rotational force produced by the seconddisk drive. In some embodiments, the rotational force produced by thesecond disk drive is opposite the rotational force produced by the firstdisk drive.

The method can include operatively coupling an information processingunit to the enclosure, and adding a memory to the enclosure, wherein theinformation processing unit is operatively coupled to the disk drivesand to the memory, wherein the information processing unit sends readcommands to the disk drives and the receives data from the disk drivesand memory. In some embodiments, a multi-processor supercomputer is usedas the information processing unit. In some embodiments, a plurality ofsubstantially similar enclosures are operatively coupled to thesupercomputer, wherein each enclosure holds a substrate and plurality ofdisk drives including at least a first disk drive and a second diskdrive positioned such that a rotational force produced by the first diskdrive is conveyed primarily as a translational force to the second diskdrive, and wherein the plurality of enclosures are operatively coupled.

In some embodiments, the method includes storing data from the diskdrives into a memory, and streaming video information from theenclosure, wherein the streaming video information includes receivinginformation from the memory and transmitting digital video to aplurality of destinations and users.

In some embodiments, the method includes causing a seek operation thatresults in a rotational force produced by the first disk drive. In someembodiments, the method includes positioning the plurality of diskdrives such that a number of the first disk drives, the number beinggreater than two and fewer than about one-hundred-and-one, are in afirst rotating orientation, and a substantially equal number of seconddisk drives are in a second counter-rotating orientation, wherein aplurality of the first disk drives and a plurality of the second diskdrives are interleaved in mechanically coupled pairs. In someembodiments, the method includes positioning the plurality of diskdrives such that a number of the first disk drives, which is greaterthan about one-hundred and fewer than about two-hundred-and-one, are ina first rotating orientation, and a substantially equal number of seconddisk drives are in a second counter-rotating orientation, wherein aplurality of the first disk drives and a plurality of the second diskdrives are interleaved in mechanically coupled pairs, each pairincluding one disk drive in a rotating orientation, and another diskdrive in a counter-rotating orientation.

In some embodiments, the method includes damping relative motion betweenat least some of the plurality of disk drives and the substrate. In someembodiments, the damping includes absorbing vibration energy in one ormore resilient materials. In some embodiments, the damping includesabsorbing vibration energy in one or more resilient materials thatinclude graded shock absorbance characteristics. In some embodiments,the damping includes absorbing vibration energy in one or more resilientmaterials that include a vibration damping polymer. In some embodiments,the damping includes absorbing vibration energy in one or more resilientmaterials that include a visco-elastic material.

In some embodiments, the method includes positioning at least some ofthe plurality of disk drives in contact with one or more boot units. Insome embodiments, the method includes providing one or more resilientmaterials for each one of the plurality of boot units. In someembodiments, the method includes providing one or more resilientmaterials for each of a plurality of boot units that include gradedshock absorbance characteristics. In some embodiments, the methodincludes providing one or more resilient materials for each of aplurality of boot units that include a vibration damping polymer. Insome embodiments, the method further includes providing one or moreresilient materials for each of a plurality of boot units that include avisco-elastic material. In some embodiments, the method includesadhesively connecting a first edge of each one of the plurality of diskdrives to its boot unit. In some embodiments, the method includesbonding a first edge of each one of the plurality of disk drives to itsboot unit.

In some embodiments, the method includes placing a detent device indisengageable contact with each one of the plurality of disk drives atan edge distal from a first edge of each one of the disk drives. In someembodiments, the method further includes sliding the detent device, thatis wedge shaped at a first end and adapted to be inserted, until itrests against each of a plurality of drives for transport and isdisengaged for disk operation. In some embodiments, the method includescamming (rotating a linear element having one or more cams for each diskdrive) the detent device that is adapted to be engaged for transport anddisengaged for disk operation.

In some embodiments, the method includes positioning the first diskdrive so that its disk rotational torque vector due to its rotatingdisk(s) is substantially antiparallel to a disk rotational torque vectorof the second disk drive that is due to its rotating disk(s). In someembodiments, the method includes positioning the first and second diskdrive such that the disk rotational torque vector of the first diskdrive is substantially collinear with the disk rotational torque vectorof the second disk drive. In some embodiments, the method includespositioning the first and second disk drive such that the diskrotational torque vector of the first disk drive is radially offset fromthe disk rotational torque vector of the second disk drive. In someembodiments, the method includes positioning the first and second diskdrive such that the actuator rotational torque vector due to actuatorarm rotation in the first disk drive is substantially collinear with theactuator rotational torque vector of the second disk drive. In someembodiments, the method includes positioning the first and second diskdrive such that the first major face of both the first and second diskdrive each have a first heat-conduction characteristic and the secondopposing major face of the first and second disk drive have a secondheat-conduction characteristic that is different from the firstheat-conduction characteristic. In some embodiments, the first majorfaces of the first and second disk drives each are substantiallymetallic. In some embodiments, the method includes positioning the firstmajor faces of the first and second disk drives such that they face eachother, wherein the first major faces of the first and second disk drivesare each portions of a respective metal cover that covers at least aportion of the respective disk drive's disk(s) and actuator arm. In someembodiments, the method includes positioning the second major faces ofthe first and second disk drives such that they face each other, whereinthe second major faces of the first and second disk drives are eachsubstantially non-metallic. In some embodiments, the method includespositioning the second major faces of the first and second disk drivessuch that they face each other, wherein the second major faces of thefirst and second disk drives each include a printed circuit board. Insome embodiments, the method includes positioning the second major facesof the first and second disk drives such that they face each other,wherein the second major faces of the first and second disk drives areeach substantially plastic.

In some embodiments, the method includes coupling the first disk driveand the second disk drive to the substrate with the first major face ofthe first disk drive facing with a partial offset the first major faceof the second disk drive. In some embodiments, the method includescoupling the first disk drive and the second disk drive to the substratewith the first major face of the first disk drive facing with no offsetthe first major face of the second disk drive. In some embodiments, themethod includes forming a first coupled pair that includes the first andsecond disk drive, and forming a second coupled pair having a third andfourth disk drive with a first major face of the third disk drive facingwith no offset a first major face of the fourth disk drive, and a secondmajor face of the second disk drive facing with partial offset a secondmajor face of the third disk drive.

In some embodiments, the method includes installing a controller thatreceives a disk access request specifying a data length of 2L and basedon the request sends substantially simultaneous disk access requests tothe first and second disk drive each specifying a data length of L. Insome embodiments, the substantially simultaneous disk access requestsent to the first and second disk drives cause seek operations havingrotational forces that at least partially cancel each other.

In some embodiments, the method includes forming the plurality of diskdrives into coupled pairs having substantially opposite rotationaltorque within each pair. In some embodiments, the method includescoupling a first edge of each coupled pair to the substrate and couplingan opposing second edge to an elastomeric material. In some embodiments,the method includes stabilizing at least some of the plurality of diskdrives with a stabilizing member having an elastomeric material incontact with the disk drives between the first edge and the second edgeof the respective disk drives. In some embodiments, the stabilizermember is a plate member having an elastomeric material in contact withat least some of the plurality of the disk drives between the first edgeand the second edge of the respective disk drives. In some embodiments,the plate member is substantially parallel to the first edge of the diskdrives and includes a plate having perforations that encircle each diskdrive. In some embodiments, the plate member further includes anelastomeric material bridging a gap between an edge of a perforation inthe plate member and the disk drive encircled by the perforation.

In some embodiments, the method includes orienting the plurality offirst disk drives and second disk drives as alternately facing coupledpairs. In some embodiments, for each one of a plurality of disk drives,a first major face of a respective drive is spaced closer to its nearestneighbor's first major face as compared to the spacing of the respectivedrive's second major face to its nearest neighbor's second major face,the second major faces having lower heat conductivity than the firstmajor faces. In some embodiments, for each one of a plurality of diskdrives, a first major face of a respective drive is spaced further fromits nearest neighbor's first major face as compared to the spacing ofthe respective drive's second major face to its nearest neighbor'ssecond major face, the second major faces having lower heat conductivitythan the first major faces.

In some embodiments, the method includes coupling each of the pluralityof first and second disk drives electrically and mechanically to thesubstrate in a row that conforms to a line, wherein the first diskdrives and the second disk drives are alternately positioned within therow as neighboring disk drives. In some embodiments, the row includestwo or more disk drives and fewer than about two-hundred-and-one diskdrives. In some embodiments, each of the first disk drives have a firstmajor face and a second opposing major face and wherein each of thesecond disk drives have a first major face and a second opposing majorface, and wherein the first major face of each first disk drive facesthe first major face of an adjoining second disk drive, and the secondmajor face of each first disk drive faces the second major face of anadjoining second disk drive. In some embodiments, the method includesconforming the row to a substantially linear line. In some embodiments,the method includes conforming the row to a substantially stepped curvedline. In some embodiments, the method includes conforming the steppedcurved line so that it follows a substantially exponential curve. Insome embodiments, the method includes conforming the row to asubstantially smooth curved line. In some embodiments, the methodincludes conforming the substantially smooth curved line so that itcurves in a substantially exponential manner. In some embodiments, themethod includes positioning the first and second disk drives with aspacing between adjacent drives, wherein the spacing between theneighboring disk drives follows a substantially exponential function. Insome embodiments, the method includes adding one or more additional rowsof disk drives. In some embodiments, the method includes positioning therows on the substrate with substantially mirror image orientationrelative to an adjoining row.

In some embodiments, the method includes elastomerically coupling thedisk drives at an edge of each disk drive that is opposite thesubstrate. In some embodiments, the method includes enclosing thesubstrate and the disk drives. In some embodiments, the substrate isoriented so that it is substantially parallel to a first major surfaceof the enclosure. In some embodiments, the method includes providing atleast one air inlet along a first surface of the enclosure and at leastone air outlet along a second surface of the enclosure. In someembodiments, the method includes adding at least one manifold thatdirects airflow over the disk drives. In some embodiments, the methodincludes flowing air through the at least one manifold and between thedisk drives. In some embodiments, the method includes adding at leastone air-movement device to the enclosure. In some embodiments, themethod includes adding one or more pairs of fans that are coupled tohave opposite rotational direction. In some embodiments, the methodincludes providing a cover for the enclosure. In some embodiments, themethod includes attaching a resilient material to the cover and to asecond edge of each one of a plurality of the disk drives. In someembodiments, the method includes attaching stiffening ribs to the cover.In some embodiments, the method includes adding a shipping-overshockdisplay. In some embodiments, the method includes adding a mother board,a personality board, or any combination thereof.

In some embodiments, the invention provides an apparatus that includesan enclosure that includes a substrate, a means in the enclosure formounting a plurality of disk drives to the enclosure, and a means forcoupling a plurality of disk drives electrically and mechanically to thesubstrate, the plurality of disk drives including at least a first and asecond disk drive, and wherein the first disk drive is positionedrelative to the second disk drive so that a rotational force produced bythe first disk drive is at least partially counteracted by a rotationalforce produced by the second disk drive.

In some embodiments, the invention provides an apparatus that includes asubstrate, and a plurality of disk drives each coupled electrically andmechanically to the substrate, the plurality of disk drives including atleast a first disk drive and a second disk drive, wherein the first andsecond disk drive each have a first major face surrounded by a first,second, third and fourth edge and having a first, second, third andfourth corner, wherein the first disk drive and the second disk driveare positioned such that a rotational force produced by the first diskdrive is conveyed primarily as a translational force to the second diskdrive. In some embodiments, the apparatus includes an enclosure, whereinthe substrate and the plurality of disk drives are attached to theenclosure, at least one memory, and an information processing unitoperatively coupled to the disk drives and to the memory, wherein theinformation processing unit sends read commands to the disk drives andreceives data from the disk drives and from the memory. In someembodiments, the information processing unit includes a multi-processorsupercomputer.

In some embodiments, the apparatus includes a plurality of substantiallysimilar enclosures, wherein each enclosure holds a substrate andplurality of disk drives including at least a first disk drive and asecond disk drive positioned such that a rotational force produced bythe first disk drive is conveyed primarily as a translational force tothe second disk drive, and wherein the plurality of enclosures areoperatively coupled to the supercomputer.

In some embodiments, the apparatus includes a memory, and avideo-streaming apparatus operatively coupled to receive data from thememory, wherein the video-streaming apparatus is adapted to transmitdigital video to a plurality of destinations and users.

In some embodiments, the apparatus includes an enclosure to which thesubstrate is connected that encloses the substrate and the plurality ofdisk drives.

In some embodiments of the apparatus, the first edge of each of thefirst and second disk drives includes a substantially neutral position,relative to rotational force, located along the first edge between thefirst corner and the second corner. In some embodiments, the first diskdrive and the second disk drive are positioned relative to each other sothat the neutral position of the first disk drive is at a position alongthe first edge of the first disk drive that is closest to the firstcorner of the second disk drive. In some embodiments, the first diskdrive and the second disk drive are positioned with their first majorfaces substantially perpendicular to each other. In some embodiments,the first disk drive and the second disk drive are positioned with theirfirst major faces at an acute angle. In some embodiments, the first diskdrive and the second disk drive are positioned with their first majorfaces substantially parallel to each other. In some embodiments, thefirst disk drive and the second disk drive are positioned with theirfirst major faces laterally offset from each other.

In some embodiments, the apparatus includes an air-deflection vanepositioned to direct additional air between the first disk drive and thesecond disk drive.

In some embodiments of the apparatus, the first disk drive and thesecond disk drive are positioned such that a rotational force producedby the second disk drive is at least partially conveyed as atranslational force to the first disk drive. In some embodiments, thefirst disk drive and the second disk drive are positioned such that therotational force produced by the second disk drive is conveyed primarilyas a translational force to the first disk drive. In some embodiments,the first disk drive and the second disk drive are positioned such thatthe rotational force produced by the first disk drive is conveyed onlyas a translational force to the second disk drive. In some embodiments,the first disk drive and the second disk drive are positioned such thatthe rotational force produced by the second disk drive is conveyed onlyas a translational force to the first disk drive.

In some embodiments of the apparatus, the first disk drive has a diskrotational torque vector due to its rotating disk(s) that issubstantially antiparallel to a disk rotational torque vector of thesecond disk drive that is due to its rotating disk(s). In someembodiments, the first disk drive and the second disk drive arepositioned with their first major faces laterally offset from eachother.

In some embodiments of the apparatus, the first disk drive has a diskrotational torque vector due to its rotating disk(s) that issubstantially coparallel (i.e., that is collinear or parallel) to a diskrotational torque vector of the second disk drive that is due to itsrotating disk(s). In some embodiments, the first disk drive and thesecond disk drive are positioned with their first major faces laterallyoffset from each other.

In some embodiments, the apparatus includes a resilient boot unitcoupled between the first edge of each of the plurality of drives andthe substrate. In some embodiments, the resilient boot unit includes avisco-elastic polymer material. In some embodiments, the resilient bootunit includes an elastomeric polymer material. In some embodiments, theapparatus includes one or more resilient materials between at least someof the plurality of disk drives and the substrate. In some embodiments,the resilient material has graded shock absorbance characteristics. Insome embodiments, the resilient material includes a visco-elasticmaterial. In some embodiments, the resilient material includes avibration damping polymer.

In some embodiments, the apparatus includes a cover plate, and aresilient cap coupled between the second edge of each of the pluralityof drives and the cover plate, wherein the second edge is on an oppositeside of the first major face from the first edge. In some embodiments,the resilient cap includes a visco-elastic polymer material. In someembodiments, the resilient cap includes an elastomeric polymer material.In some embodiments, the resilient cap is adhesively coupled to thesecond edge of at least some of the plurality of disk drives. In someembodiments, the resilient boot unit is adhesively coupled to the firstedge of at least some of the plurality of disk drives.

In some embodiments of the apparatus, the enclosure includes at leastone air-inlet manifold and at least one air-outlet manifold, wherein airsubstantially passes from the inlet manifold between the first diskdrive and second disk drive to the outlet manifold. In some embodiments,the apparatus includes at least one manifold that directs airflow overthe disk drives. In some embodiments, the apparatus includes anair-movement device. In some embodiments, the air-movement deviceincludes one or more fans. In some embodiments, the air-movement deviceincludes at least one pair of fans that are mechanically coupled andhave opposite rotation directions.

The invention provides a method that includes mounting a plurality ofdisk drives in an enclosure, the plurality of disk drives including atleast a first disk drive and a second disk drive that are eachelectrically and mechanically coupled to the enclosure, and mechanicallycoupling the first disk drive and the second disk drive such thatrotational force produced by the first disk drive is at least partiallytransmitted as translational force to the second disk drive. In someembodiments, the method includes operatively coupling an informationprocessing unit to the enclosure, and adding a memory to the enclosure,wherein the information processing unit is operatively coupled to thedisk drives and to the memory, and wherein the information processingunit sends read commands to the disk drives and receives data from thedisk drives and memory. In some embodiments, the method includesutilizing a multi-processor supercomputer as the information processingunit. In some embodiments, the method includes operatively coupling aplurality of substantially similar enclosures to the supercomputer,wherein each enclosure holds a substrate and plurality of disk drivesincluding at least a first disk drive and a second disk drive positionedsuch that a rotational force produced by the first disk drive isconveyed primarily as a translational force to the second disk drive,and wherein the plurality of enclosures are operatively coupled. In someembodiments, the method includes operatively coupling a memory to theenclosure, and operatively coupling a video-streaming apparatus to theenclosure, wherein the video-streaming apparatus receives data from thememory and is adapted to transmit digital video to a plurality ofdestinations and users.

In some embodiments, the method includes performing a seek function withthe first disk drive, wherein a rotational force is produced.

In some embodiments, the method includes positioning the first diskdrive and the second disk drive relative to each other so that a neutralposition of the first disk drive is positioned along a first edge of thefirst disk drive that is closest to a first corner of the second diskdrive. In some embodiments, the method includes positioning the firstdisk drive and the second disk drive with their first major facessubstantially perpendicular to each other. In some embodiments, themethod includes positioning the first disk drive and the second diskdrive with their first major faces at an acute angle to each other. Insome embodiments, the method includes positioning the first disk driveand the second disk drive with their first major faces substantiallyparallel to each other. In some embodiments, the method includespositioning the first disk drive and the second disk drive with theirfirst major faces laterally offset from each other.

In some embodiments, the method includes positioning an air-deflectionvane to direct additional air between the first disk drive and thesecond disk drive.

In some embodiments, the method includes positioning the first diskdrive and the second disk drive such that a rotational force produced bythe second disk drive is at least partially conveyed as a translationalforce to the first disk drive. In some embodiments, the method includespositioning the first disk drive and the second disk drive such that therotational force produced by the second disk drive is substantiallyconveyed as a translational force to the first disk drive. In someembodiments, the method includes positioning the first disk drive andthe second disk drive such that the rotational force produced by thefirst disk drive is conveyed only as a translational force to the seconddisk drive. In some embodiments, the method includes positioning thefirst disk drive and the second disk drive such that the rotationalforce produced by the second disk drive is conveyed only as atranslational force to the first disk drive. In some embodiments, themethod includes positioning the first disk drive so that a diskrotational torque vector due to its rotating disk(s) is substantiallyantiparallel to a disk rotational torque vector of the second disk drivethat is due to its rotating disk(s). In some embodiments, the method ofclaim 5, further including positioning the first disk drive and thesecond disk drive with their first major faces laterally offset fromeach other.

In some embodiments, the method includes positioning the first diskdrive so that a disk rotational torque vector due to its rotatingdisk(s) is substantially coparallel to a disk rotational torque vectorof the second disk drive that is due to its rotating disk(s). In someembodiments, the method includes positioning the first disk drive andthe second disk drive with their first major faces laterally offset fromeach other. In some embodiments, the method includes damping relativemotion between at least some of the plurality of disk drives and thesubstrate. In some embodiments, the method includes using one or moreresilient materials to dampen vibration energy. In some embodiments, themethod includes using one or more resilient materials having gradedshock absorbance characteristics. In some embodiments, the methodincludes using one or more resilient materials that include a vibrationdamping polymer. In some embodiments, the method includes using one ormore resilient materials that include a visco-elastic material. In someembodiments, the method includes coupling a resilient boot unit betweenthe first edge of each of the plurality of drives and the substrate. Insome embodiments, the method includes using a resilient boot unit thatincludes a visco-elastic polymer material. In some embodiments, themethod includes using a resilient boot unit that includes an elastomericpolymer material.

In some embodiments, the method includes adding a cover plate, andcoupling a resilient cap between the second edge of each of theplurality of drives and the cover plate. In some embodiments, the methodincludes using a resilient cap that includes a visco-elastic polymermaterial. In some embodiments, the method includes using a resilient capthat includes an elastomeric polymer material. In some embodiments, themethod includes adhesively coupling the resilient cap to the second edgeof at least some of the plurality of disk drives. In some embodiments,the method includes adhesively coupling the resilient boot unit to thefirst edge of at least some of the plurality of disk drives.

The invention provides an apparatus that includes a substrate, and ameans for mounting a plurality of disk drives to the substrate, and ameans for coupling a plurality of disk drives electrically andmechanically to the substrate, the plurality of disk drives including atleast a first disk drive and a second disk drive, wherein the first andsecond disk drive each have a first major face surrounded by a first,second, third and fourth edge and having a first, second, third andfourth corner, wherein the first disk drive and the second disk driveare positioned such that a rotational force produced by the first diskdrive is conveyed primarily as a translational force to the second diskdrive.

The invention provides an apparatus that includes a substrate, and aplurality of disk-drive connectors each coupled electrically andmechanically to the substrate, the plurality of disk-drive connectorsincluding at least a first and a second disk-drive connector, whereinthe first disk-drive connector is positioned relative to the seconddisk-drive connector so that a rotational force produced by a first diskdrive that is connected to the first disk-drive connector is at leastpartially counteracted by a rotational force produced by a second diskdrive that is connected to the second disk-drive connector. In someembodiments, the apparatus includes an enclosure, wherein the substrateand the plurality of disk-drive connectors are attached to theenclosure, at least one memory, and an information processing unitoperatively coupled to the disk-drive connectors and to the memory,wherein the information processing unit sends read commands to diskdrives that are connected to the disk-drive connectors and receives datafrom the disk drives and from the memory. In some embodiments, theinformation processing unit includes a multi-processor supercomputer. Insome embodiments, the apparatus includes a plurality of substantiallysimilar enclosures, wherein each enclosure holds a substrate andplurality of disk-drive connectors including at least a first disk-driveconnector and a second disk-drive connector positioned such that arotational force produced by a first disk drive that is connected to thefirst disk-drive connector is conveyed primarily as a translationalforce to a second disk drive that is connected to the second disk-driveconnector, and wherein the plurality of enclosures are operativelycoupled to the supercomputer. In some embodiments, the apparatusincludes a memory, and a video-streaming apparatus operatively coupledto receive data from the memory, wherein the video-streaming apparatusis adapted to transmit digital video to a plurality of destinations andusers.

In some embodiments of the apparatus, the plurality of disk-driveconnectors includes more than two first disk-drive connectors in a firstrotating orientation and fewer than about one-hundred-and-one firstdisk-drive connectors, and a substantially equal number of seconddisk-drive connectors in a second counter rotating orientation, whereina plurality of the first and a plurality of the second disk-driveconnectors are interleaved in coupled pairs. In some embodiments, theplurality of disk-drive connectors includes more than about one-hundredfirst disk-drive connectors and fewer than about two-hundred-and-onefirst disk-drive connectors, and a substantially equal number of seconddisk-drive connectors, wherein a plurality of the first and a pluralityof the second disk-drive connectors are interleaved in coupled pairs sothat first disk drives connected to the first disk-drive connectors eachrotate in one orientation and the second disk drives connected to thesecond disk-drive connectors rotate in a counter orientation. In someembodiments of the apparatus, at least some of the plurality ofdisk-drive connectors are each in contact with a boot unit. In someembodiments, the boot unit includes one or more resilient materials. Insome embodiments, the boot unit has graded shock absorbancecharacteristics. In some embodiments, the boot unit includes a vibrationdamping polymer. In some embodiments, the boot unit includes avisco-elastic material. In some embodiments of the apparatus, at least aportion of each one of the plurality of disk-drive connectors isadhesively connected to its boot unit. In some embodiments, of theapparatus, at least a portion of each one of the plurality of disk-driveconnectors is bonded to its boot unit.

In some embodiments, the apparatus includes a detent device adapted tobe placed in disengageable contact with each one of a plurality of diskdrives that are connected to each one of the disk-drive connectors,wherein the detent device contacts the disk drive at an edge distal fromthe disk drive's first edge. In some embodiments, the detent device iswedge shaped at a first end and adapted to be inserted against each of aplurality of disk drives that are inserted into the disk-driveconnectors, wherein the detent device can be used for transport anddisengaged for disk drive operation. In some embodiments, the detentdevice includes a cam mechanism adapted to be engaged for transport anddisengaged for operation of disk drives that are connected to each ofthe disk-drive connectors.

In some embodiments of the apparatus, the first disk-drive connector ispositioned so that a first disk drive that is connected to the firstdisk-drive connector has a disk rotational torque vector due to itsrotating disk(s) that is substantially antiparallel to a disk rotationaltorque vector that is due to a rotating disk(s) of a second disk drivethat is connected to the second disk-drive connector. In someembodiments of the apparatus, the disk rotational torque vector of thefirst disk drive that is connected to the first disk-drive connector issubstantially collinear with the disk rotational torque vector of thesecond disk drive that is connected to the second disk-drive connector.In some embodiments of the apparatus, the disk rotational torque vectorof the first disk drive that is mounted in the first disk-driveconnector is radially offset from the disk rotational torque vector ofthe second disk drive that is connected to the second disk-driveconnector. In some embodiments of the apparatus, the actuator rotationaltorque vector due to actuator arm rotation in a first disk drive that ismounted in the first disk-drive connector is substantially collinearwith the actuator rotational torque vector of a second disk drive thatis connected to the second disk-drive connector. In some embodiments ofthe apparatus, the first disk-drive connector and the second disk-driveconnector are coupled to the substrate so that a first disk driveconnected to the first disk-drive connector is oriented with a firstmajor face of the first disk drive facing with a partial offset a firstmajor face of a second disk drive that is mounted in the seconddisk-drive connector. In some embodiments, the first disk-driveconnector and the second disk-drive connector are coupled to thesubstrate so that a first disk drive connected to the first disk-driveconnector is oriented with a first major face of the first disk drivefacing with no offset a first major face of a second disk drive that ismounted in the second disk-drive connector. In some embodiments, thefirst and second disk-drive connectors form a first coupled pair,further comprising a second coupled pair having a third and fourthdisk-drive connector, wherein the disk-drive connectors are positionedso that a first major face of a third disk drive connected to the thirddisk-drive connector faces with no offset a first major face of a fourthdisk drive that is connected to the fourth disk-drive connector, and asecond major face of a second disk drive that is connected to the seconddisk-drive connector faces with partial offset a second major face ofthe third disk drive that is connected to the third disk-driveconnector.

In some embodiments, the apparatus includes a controller that receives adisk access request specifying a data length of 2L and based on therequest sends substantially simultaneous disk access requests to a firstand second disk drive that each specify a data length of L, wherein thefirst and second disk drive are each connected to a first and seconddisk-drive connector. In some embodiments, the substantiallysimultaneous disk access request sent to the first and second diskdrives cause seek operations having rotational forces that at leastpartially cancel each other.

In some embodiments of the apparatus, the plurality of disk-driveconnectors are formed into coupled pairs so that disk drives connectedto the disk-drive connectors have substantially opposite rotationaltorque within each pair. In some embodiments, a portion of each coupledpair of disk-drive connectors is coupled to the substrate and a portionof each disk-drive connector is coupled to an elastomeric material.

In some embodiments, the apparatus includes a stabilizer member havingan elastomeric material in contact with at least a portion of thedisk-drive connectors. In some embodiments, the stabilizer member is aplate member having an elastomeric material in contact with at least aportion of the disk-drive connectors.

In some embodiments of the apparatus, the plurality of first disk driveand second disk-drive connectors are oriented as alternately facingcoupled pairs. In some embodiments, the plurality of disk-driveconnectors are positioned as a first pair of disk-drive connectors thatinclude first and second disk-drive connectors and a second pair ofdisk-drive connectors that include third and fourth disk-driveconnectors on the substrate, wherein a space between the first andsecond disk-drive connectors is less than a space between the first andsecond pairs of disk-drive connectors.

In some embodiments of the apparatus, the plurality of first and seconddisk-drive connectors are each coupled electrically and mechanically tothe substrate in a row that conforms to a line, wherein the firstdisk-drive connectors and the second disk-drive connectors are facing inalternate directions positioned within the row. In some embodiments, therow includes two or more disk-drive connectors and fewer thantwo-hundred-and-one disk-drive connectors. In some embodiments, the rowconforms to a substantially linear line. In some embodiments, the rowconforms to a substantially stepped curved line. In some embodiments,the stepped curved line curves in a substantially exponential manner. Insome embodiments, the row conforms to a substantially smooth curvedline. In some embodiments, the substantially smooth curved line curvesin a substantially exponential manner.

In some embodiments, the apparatus includes one or more additional rowsof disk-drive connectors. In some embodiments, the rows are positionedon the substrate with substantially mirror image orientation relative toa neighboring row.

In some embodiments, the apparatus includes an enclosure. In someembodiments, the substrate is oriented parallel to a first major surfaceof the enclosure. In some embodiments, the enclosure includes at leastone air inlet and at least one air outlet. In some embodiments, theapparatus includes at least one manifold that directs airflow over diskdrives when they are connected to the disk-drive connectors. In someembodiments, the apparatus includes an air-movement device. In someembodiments, the air-movement device includes one or more fans. In someembodiments, the air-movement device includes one or more pairs of fansthat rotate in opposite directions. In some embodiments, the enclosureincludes a cover. In some embodiments, the cover includes a resilientmaterial. In some embodiments, a resilient material is attached to thecover. In some embodiments, the cover includes at least one stiffeningrib. In some embodiments, the apparatus includes a resilient materialthat is attached to a second edge of each one of a plurality of diskdrives that are connected to the disk-drive connectors. In someembodiments, the apparatus includes a shipping-overshock display. Insome embodiments, the apparatus includes a mother board, a personalityboard, or any combination thereof.

The invention provides an apparatus that includes a substrate, and aplurality of disk-drive connectors each coupled electrically andmechanically to the substrate, the plurality of disk-drive connectorsincluding at least a first disk-drive connector and a second disk-driveconnector, wherein the first disk-drive connector and the seconddisk-drive connector are positioned such that a rotational forceproduced by a first disk drive connected to the first disk-driveconnector is conveyed primarily as a translational force to a seconddisk drive connected to the second disk-drive connector. In someembodiments, the apparatus includes an enclosure, wherein the substrateand the plurality of disk-drive connectors are attached to theenclosure, at least one memory, and an information processing unitoperatively coupled to disk drives that are connected to the disk-driveconnectors and to the memory, wherein the information processing unitsends read commands to the disk drives and receives data from the diskdrives and from the memory. In some embodiments, the informationprocessing unit includes a multi-processor supercomputer.

In some embodiments, the apparatus includes a plurality of substantiallysimilar enclosures, wherein each enclosure holds a substrate andplurality of disk-drive connectors including at least a first disk-driveconnector and a second disk-drive connector that are positioned suchthat a rotational force produced by a first disk drive connected to thefirst disk-drive connector is conveyed primarily as a translationalforce to a second disk drive that is connected to the second disk-driveconnector, and wherein the plurality of enclosures are operativelycoupled to the supercomputer.

In some embodiments, the apparatus includes a memory, and avideo-streaming apparatus operatively coupled to receive data from thememory, wherein the video-streaming apparatus is adapted to transmitdigital video to a plurality of destinations and users.

In some embodiments, the apparatus includes an enclosure to which thesubstrate is connected that encloses the substrate and a plurality ofdisk drives that are connected to the plurality of disk-driveconnectors.

In some embodiments of the apparatus, the disk-drive connectors arepositioned so that a first edge of each of a first and second disk drivethat are connected to the disk-drive connectors includes a substantiallyneutral position, relative to rotational force, located along the firstedge between the first corner and the second corner of the disk drive.In some embodiments, the first disk-drive connector and the seconddisk-drive connector are positioned relative to each other so that theneutral position of a first disk drive connected to the first disk-driveconnector is at a position along the first edge of a first disk drivethat is closest to the first corner of a second disk drive that isconnected to the second disk-drive connector. In some embodiments, thefirst disk-drive connector and the second disk-drive connector arepositioned so that a first and second disk drives connected to the firstand second disk-drive connectors are positioned with their first majorfaces substantially perpendicular to each other. In some embodiments,the first disk-drive connector and the second disk-drive connector arepositioned so that a first disk drive connected to the first disk-driveconnector and a second disk drive connected to the second disk-driveconnector are positioned with their first major faces at an acute angle.In some embodiments, the first disk-drive connector and the seconddisk-drive connector are positioned so that a first disk drive connectedto the first disk-drive connector and a second disk drive connected tothe second disk-drive connector are positioned with their first majorfaces substantially parallel to each other. In some embodiments, thefirst disk-drive connector and the second disk-drive connector arepositioned so that a first disk drive connected to the first disk-driveconnector and a second disk drive connected to the second disk-driveconnector are positioned with their first major faces laterally offsetfrom each other. In some embodiments, the first disk-drive connector andthe second disk-drive connector are also positioned such that arotational force produced by a second disk drive that is connected tothe second disk-drive connector is at least partially conveyed as atranslational force to a first disk drive that is connected to the firstdisk-drive connector. In some embodiments, the first disk-driveconnector and the second disk-drive connector are also positioned suchthat the rotational force produced by a second disk drive that isconnected to the second disk-drive connector is conveyed primarily as atranslational force to a first disk drive that is connected to a firstdisk-drive connector. In some embodiments, the first disk-driveconnector and the second disk-drive connector are also positioned suchthat a rotational force produced by a first disk drive that is connectedto a first disk drive is conveyed only as a translational force to asecond disk drive that is connected to the second disk-drive connector.In some embodiments, the first disk-drive connector and the seconddisk-drive connector are also positioned such that a rotational forceproduced by a second disk drive that is connected to the seconddisk-drive connector is conveyed only as a translational force to afirst disk drive that is connected to the first disk-drive connector.

In some embodiments of the apparatus, the first disk drive has a diskrotational torque vector due to its rotating disk(s) that issubstantially antiparallel to a disk rotational torque vector of thesecond disk drive that is due to its rotating disk(s). In someembodiments, the first disk-drive connector and the second disk-driveconnector are positioned so that a first disk drive that is connected tothe first disk-drive connector and a second disk drive that is connectedto the second disk-drive connector are positioned with their first majorfaces laterally offset from each other. In some embodiments, the firstdisk-drive connector and the second disk-drive connector are positionedso that a first disk drive that is connected to the first disk-driveconnector has a disk rotational torque vector due to its rotatingdisk(s) that is substantially coparallel to a disk rotational torquevector due to a rotating disk(s) of a second disk drive that isconnected to the second disk-drive connector. In some embodiments, thefirst disk-drive connector and the second disk-drive connector arepositioned so that a first disk drive that is connected to the firstdisk-drive connector and a second disk drive that is connected with thesecond disk-drive connector have their first major faces laterallyoffset from each other.

In some embodiments, the apparatus includes one or more resilientmaterials between at least some of the plurality of disk-driveconnectors and the substrate. In some embodiments, the resilientmaterial has graded shock absorbance characteristics. In someembodiments, the resilient material includes a visco-elastic material.In some embodiments, the resilient material includes a vibration dampingpolymer.

In some embodiments, the apparatus includes an enclosure that includesat least one air-inlet manifold and at least one air-outlet manifold,wherein air substantially passes from the inlet manifold between a firstdisk drive that is connected to the first disk-drive connector and asecond disk drive that is connected to the second disk-drive connectorto the outlet manifold.

In some embodiments, the apparatus includes an air-movement device. Insome embodiments, the air-movement device includes one or more fans. Insome embodiments, the air-movement device includes at least one pair offans that are mechanically coupled and have opposite rotationdirections.

The invention provides a method that includes mounting a plurality ofdisk-drive connectors in an enclosure, the enclosure including aconnector substrate, the plurality of disk-drive connectors including atleast a first disk-drive connector and a second disk-drive connectorthat are each electrically and mechanically coupled to the enclosure,and mechanically coupling the first disk-drive connector and the seconddisk-drive connector such that rotational force produced by a first diskdrive that is connected to the first disk-drive connector is at leastpartially counteracted by rotational force produced by a second diskdrive that is connected to the second disk-drive connector.

In some embodiments, the method includes operatively coupling aninformation processing unit to the enclosure, and adding a memory to theenclosure, wherein the information processing unit is operativelycoupled to disk drives that are connected to the disk-drive connectorsand to the memory, wherein the information processing unit sends readcommands to the disk drives and receives data from the disk drives andmemory. In some embodiments, the method includes utilizing amulti-processor supercomputer as the information processing unit.

In some embodiments, the method includes operatively coupling aplurality of substantially similar enclosures to the supercomputer,wherein each enclosure holds a substrate and plurality of disk-driveconnectors including at least a first disk-drive connector and a seconddisk-drive connector positioned such that a rotational force produced bya first disk drive that is connected to the first disk-drive connectoris conveyed primarily as a translational force to a second disk drivethat is connected to the second disk-drive connector, and wherein theplurality of enclosures are operatively coupled.

In some embodiments, the method includes operatively coupling a memoryto the enclosure, and operatively coupling a video-streaming apparatusto the enclosure, wherein the video-streaming apparatus receives datafrom the memory and is adapted to transmit digital video to a pluralityof destinations and users.

In some embodiments, the method includes positioning the plurality ofdisk-drive connectors such that a number of the first disk-driveconnectors, the number being greater than two and fewer than aboutone-hundred-and-one, are in a first orientation, and a substantiallyequal number of second disk-drive connectors are in a secondorientation, wherein a plurality of first disk drives and second diskdrives that are connected to the first and second disk-drive connectorsare interleaved in mechanically coupled pairs with opposite rotatingorientation. In some embodiments, the method includes positioning thefirst disk-drive connector so that a disk rotational torque vector dueto a rotating disk(s) of a first disk drive that is connected to thefirst disk-drive connector is substantially antiparallel to a diskrotational torque vector of a second disk drive that is due to arotating disk(s) of a second disk drive that is connected to the seconddisk-drive connector. In some embodiments, the method includespositioning the first disk-drive connector so that a disk rotationaltorque vector due to a rotating disk(s) of a first disk drive that isconnected to the first disk-drive connector is substantially collinearto a disk rotational torque vector of a second disk drive that is due toa rotating disk(s) of a second disk drive that is connected to thesecond disk-drive connector. In some embodiments, the method includespositioning the first disk-drive connector so that a disk rotationaltorque vector due to a rotating disk(s) of a first disk drive that isconnected to the first disk-drive connector is radially offset to a diskrotational torque vector of a second disk drive that is due to arotating disk(s) of a second disk drive that is connected to the seconddisk-drive connector. In some embodiments, the method includespositioning the first disk-drive connector so that a disk rotationaltorque vector due to a rotating disk(s) of a first disk drive that isconnected to the first disk-drive connector is collinear to a diskrotational torque vector of a second disk drive that is due to arotating disk(s) of a second disk drive that is connected to the seconddisk-drive connector. In some embodiments, the method includes couplingthe first disk-drive connector and the second disk-drive connector tothe substrate so that a first major face of a first disk drive connectedto the first disk-drive connector faces with a partial offset of a firstmajor face of a second disk drive that is connected to the seconddisk-drive connector. In some embodiments, the method includes couplingthe first disk-drive connector and the second disk-drive connector tothe substrate so that a first major face of a first disk drive connectedto the first disk-drive connector faces with no offset of a first majorface of a second disk drive that is connected to the second disk-driveconnector. In some embodiments, the method includes forming a firstcoupled pair that includes the first and second disk-drive connector,and forming a second coupled pair having a third and fourth disk-driveconnector so that a first major face of a third disk drive that isconnected to the third disk-drive connector faces with no offset a firstmajor face of a fourth disk drive that is connected to the fourthdisk-drive connector, and a second major face of a second disk drivethat is connected to the second disk-drive connector faces with partialoffset a second major face of the third disk drive.

In some embodiments, the method includes installing a controller thatreceives a disk access request specifying a data length of 2L and basedon the request sends substantially simultaneous disk access requestsspecifying a data length of L a first and second disk drive that areconnected to the first and second disk-drive connectors. In someembodiments of the method, the substantially simultaneous disk accessrequest sent to the first and second disk drives cause seek operationshaving rotational forces that at least partially cancel each other.

In some embodiments, the method includes forming the plurality ofdisk-drive connectors into coupled pairs so that disk drives connectedto the disk-drive connectors have substantially opposite rotationaltorque within each pair of disk drives. In some embodiments, the methodincludes orienting the plurality of first disk-drive connectors andsecond disk-drive connectors so that first disk drives and second diskdrives connected to the first and second disk-drive connectors formalternately facing coupled pairs. In some embodiments, the methodincludes coupling each of the plurality of first and second disk-driveconnectors electrically and mechanically to the substrate in a row thatconforms to a line, wherein first and second disk drives that areconnected to the first and second disk-drive connectors are alternatelypositioned within the row as neighboring disk drives. In someembodiments, the row includes two or more disk-drive connectors andfewer than about two-hundred-and-one disk-drive connectors. In someembodiments, the method includes conforming the row to a substantiallylinear line. In some embodiments, the method includes conforming the rowto a substantially stepped curved line. In some embodiments, the methodincludes conforming the stepped curved line so that it follows asubstantially exponential curve. In some embodiments, the methodincludes conforming the row to a substantially smooth curved line. Insome embodiments, the method includes conforming the substantiallysmooth curved line so that it curves in a substantially exponentialmanner. In some embodiments, the method includes positioning the firstand second disk-drive connectors with a spacing between adjacentdisk-drive connectors, wherein the spacing between neighboringdisk-drive connectors follows a substantially exponential function. Insome embodiments, the method includes adding one or more additional rowsof disk-drive connectors. In some embodiments, the method includespositioning the rows on the substrate with substantially mirror imageorientation relative to an adjoining row.

In some embodiments, the method includes enclosing the substrate anddisk-drive connectors in an enclosure. In some embodiments, the methodincludes orienting the substrate so that it is substantially parallel toa first major surface of the enclosure. In some embodiments, the methodincludes providing at least one air inlet along a first surface of theenclosure and at least one air outlet along a second surface of theenclosure. In some embodiments, the method includes adding at least onemanifold that directs airflow over the disk-drive connectors. In someembodiments, the method includes adding at least one air-movement deviceto the enclosure. In some embodiments, the method includes adding one ormore pairs of fans that are coupled to have opposite rotationaldirection.

In some embodiments, the method includes providing a cover for theenclosure. In some embodiments, the method includes attaching stiffeningribs to the cover. In some embodiments, the method includes adding ashipping-overshock display. In some embodiments, the method includesadding a mother board, a personality board, or any combination thereof.

In some embodiments, the invention provides an apparatus that includesan enclosure for holding a plurality of drives in each of one or morerows including a first row, a plurality of sockets arranged along thefirst row with the socket's long dimensions generally parallel to oneanother and at a non-parallel angle to the first row, each socketproviding electrical connection and mechanical support along a firstconnector edge of one or more disk drives, and a resilient supportmember adapted to hold a second edge other than the first connector edgeof each disk drive, such that the enclosure forms an inlet air manifoldalong a first side of the first row and an outlet air manifold along anopposite second side of the first row.

In some embodiments of the apparatus, the inlet air manifold has alength measured parallel to the first row that is longer than the inletair manifold's width measured perpendicular to the first row, andwherein the outlet air manifold has a length measured parallel to thefirst row that is longer than the outlet air manifold's width measuredperpendicular to the first row.

In some embodiments of the apparatus, the sockets for the first row aremounted to circuit board forming an internal plane of the enclosure, andwherein the resilient support member includes a cover mounted parallelto the circuit board.

In some embodiments of the apparatus, the cover includes a sheet-metalplate and a visco-elastic material that is located between the plate andeach disk drive position, the visco-elastic material adapted to adhereto the cover and to each disk drive.

Some embodiments of the apparatus further include a plurality of diskdrives mounted to the enclosure.

In some embodiments, the invention provides a method that includesmounting a plurality of disk-drive connectors in an enclosure, theplurality of disk-drive connectors including at least a first disk-driveconnector and a second disk-drive connector that are each electricallyand mechanically coupled to the enclosure, and mechanically coupling thefirst disk-drive connector and the second disk-drive connector such thatrotational force produced by a first disk drive that is connected to thefirst disk-drive connector is at least partially transmitted astranslational force to a second disk drive that is connected to thesecond disk-drive connector.

In some embodiments, the method includes operatively coupling aninformation processing unit to the enclosure, and adding a memory to theenclosure, wherein the information processing unit is operativelycoupled to disk drives that are connected to the disk-drive connectorsand to the memory, and wherein the information processing unit sendsread commands to the disk drives and receives data from the disk drivesand memory. In some embodiments, the method includes utilizing amulti-processor supercomputer as the information processing unit.

In some embodiments, the method includes operatively coupling a memoryto the enclosure, and operatively coupling a video-streaming apparatusto the enclosure, wherein the video-streaming apparatus receives datafrom the memory and is adapted to transmit digital video to a pluralityof destinations and users.

In some embodiments, the method includes positioning the firstdisk-drive connector and the second disk-drive connector relative toeach other so that a neutral position of the first disk drive that isconnected to the first disk-drive connector is at a position along thefirst edge of the first disk drive that is closest to a first corner ofa second disk drive that is connected to the second disk-driveconnector. In some embodiments, the method includes positioning thefirst disk-drive connector and the second disk-drive connector so that afirst disk drive connected to the first disk-drive connector and asecond disk drive connected to the second disk-drive connector arepositioned with their first major faces substantially perpendicular toeach other. In some embodiments, the method includes positioning thefirst disk-drive connector and the second disk-drive connector so that afirst disk drive connected to the first disk-drive connector and asecond disk drive connected to the second disk-drive connector arepositioned with their first major faces at an acute angle. In someembodiments, the method includes positioning the first disk-driveconnector and the second disk-drive connector so that a first disk driveconnected to the first disk-drive connector and a second disk driveconnected to the second disk-drive connector are positioned with theirfirst major faces substantially parallel to each other. In someembodiments, the method includes positioning the first disk-driveconnector and the second disk-drive connector so that a first disk driveconnected to the first disk-drive connector and a second disk driveconnected to the second disk-drive connector are positioned with theirfirst major faces laterally offset from each other.

In some embodiments, the method includes positioning an air-deflectionvane to direct additional air between a first disk drive and a seconddisk drive that are connected to the first disk-drive connector and thesecond disk-drive connector.

In some embodiments, the method includes positioning the firstdisk-drive connector and the second disk-drive connector such that arotational force produced by a second disk drive that is connected tothe second disk-drive connector is at least partially conveyed as atranslational force to a first disk drive that is connected to the firstdisk-drive connector. In some embodiments, the method includespositioning the first disk-drive connector and the second disk-driveconnector such that a rotational force produced by a second disk drivethat is connected to the second disk-drive connector is substantiallyconveyed as a translational force to a first disk drive that isconnected to the first disk-drive connector. In some embodiments, themethod includes positioning the first disk-drive connector and thesecond disk-drive connector such that a rotational force produced by asecond disk drive that is connected to the second disk-drive connectoris conveyed only as a translational force to a first disk drive that isconnected to the first disk-drive connector. In some embodiments, themethod includes positioning the first disk-drive connector and thesecond disk-drive connector so that a disk rotational torque vector dueto a rotating disk(s) of a first disk drive connected to the firstdisk-drive connector is substantially antiparallel to a disk rotationaltorque vector that is due to a rotating disk(s) of a second disk drivethat is connected to the second disk-drive connector. In someembodiments, the method includes positioning the first disk-driveconnector and the second disk-drive connector so that a first major faceof a first disk drive that is connected to the first disk-driveconnector is laterally offset from a first major face of a second diskdrive that is connected to the second disk-drive connector.

In some embodiments, the method includes positioning the firstdisk-drive connector and the second disk-drive connector so that a diskrotational torque vector due to a rotating disk(s) of a first disk driveconnected to the first disk-drive connector is substantially coparallelto a disk rotational torque vector that is due to a rotating disk(s) ofa second disk drive that is connected to the second disk-driveconnector. In some embodiments, the method includes positioning thefirst disk-drive connector and the second disk-drive connector so that afirst major face of a first disk drive that is connected to the firstdisk-drive connector is laterally offset from a first major face of asecond disk drive that is connected to the second disk-drive connector.

In some embodiments, the method includes damping relative motion betweenat least some of the plurality of disk drives that are connected to theplurality of disk-drive connectors and the substrate. In someembodiments, the method includes using one or more resilient materialsto dampen vibration energy. In some embodiments, the method includesusing one or more resilient materials having graded shock absorbancecharacteristics. In some embodiments, the method includes using one ormore resilient materials that include a vibration damping polymer. Insome embodiments, the method includes using one or more resilientmaterials that include a visco-elastic material.

The invention provides a method that includes mounting a plurality ofdisk drives in an enclosure, the enclosure including a connectorsubstrate, the plurality of disk drives including at least a first diskdrive and a second disk drive, vibrationally coupling the first diskdrive to the second disk drive, and sending a first seek operation tothe first disk drive and a second seek operation to the second diskdrive, wherein a timing of the first seek operation relative to thesecond seek operation is adjusted to minimize adverse vibrationalinteraction between the first disk drive and the second disk drive.

In some embodiments, the method includes mechanically coupling the firstdisk drive and the second disk drive such that rotational force producedby the first disk drive is at least partially counteracted by rotationalforce produced by the second disk drive. In some embodiments, the firstand second seek operations are performed substantially simultaneously.In some embodiments, the first and second seek operations are timed sothat the second seek operation does not occur while the first disk driveis reading data. In some embodiments, the first and second seekoperations are timed so that the second seek operation does not occurwhile the first disk drive is writing data.

In some embodiments, the method includes obtaining vibration-interactioninformation regarding the first and second disk drives and adjusting thetime of the second seek operation based on the information. In someembodiments, the method includes performing a plurality of seekoperations to the second disk drive while the first disk drive isreading data in order to generate the vibration-interaction information.In some embodiments, the method includes storing the vibrationinteraction information in a look-up table.

In some embodiments of the method, the plurality of disk drives furtherinclude a third disk drive and a fourth disk drive and the methodfurther includes performing a plurality of seek operations to the thirddisk drive while the first disk drive is reading data in order togenerate vibration-interaction information relating to the third andfirst disk drives, storing the vibration interaction information in thelook-up table, and choosing between performing a seek operation to thesecond disk drive versus performing a seek operation to the third diskdrive based on the vibration-interaction information contained in thelook-up table.

The invention provides an apparatus that includes a data structurehaving a plurality of entries, each entry containingvibration-interaction information relative to a read operation occurringon a first disk drive of a pair of disk drives and a seek operationbeing performed on a second disk drive of the pair. In some embodiments,the apparatus includes a memory and an information processing unitoperatively coupled together, wherein the data structure is stored inthe memory and wherein the information processing unit is adapted toadjust a timing of at least one seek operation based on informationstored in the data structure. In some embodiments, the apparatusincludes a video-streaming unit operatively coupled to the informationprocessing unit, wherein the video-streaming unit receives data from thememory and is adapted to transmit digital video to a plurality ofdestinations and users. In some embodiments, the apparatus includes amulti-processor supercomputer operatively coupled to the informationprocessing unit.

The invention provides an apparatus that includes a memory, the memoryholding vibration-interaction information, and an information processingunit operatively coupled to the memory to receive thevibration-interaction information and adjust a timing of seek operationsto a plurality of disk drives based on the information. In someembodiments, the apparatus includes an enclosure that holds theplurality of disk drives, the enclosure operatively coupled to theinformation-processing unit.

The invention provides a method that includes mounting a plurality ofdisk drives in shock mounts in an enclosure, and detenting the pluralityof disk drives against vibration using a disengagable detent device. Insome embodiments of the method, the detenting includes inserting adisengagable detent device that is wedge shaped at a first end andadapted to be inserted against each of a plurality of disk drives fortransport and which can be disengaged for disk operation. In someembodiments, the inserting includes wedging the detent device against aplurality of disk drives in a non-simultaneous sequential manner. Insome embodiments, the detenting includes camming a disengagable detentdevice into an engaged position for shipping; wherein the detent deviceis adapted to be disengaged for disk drive operation. In someembodiments, the camming is performed against a plurality of disk drivesin a non-simultaneous sequential manner. In some embodiments, thecamming is performed against a plurality of disk drives in asubstantially simultaneous manner.

The invention provides an apparatus that includes an enclosure, asubstrate held within the enclosure, a plurality of disk-driveconnectors each coupled mechanically to the substrate, the plurality ofdisk-drive connectors including at least a first and a second disk-driveconnector, and an over-shock detector operatively coupled to theenclosure and adapted to detect and store information regarding one ormore over-shock events. In some embodiments, the apparatus includes atleast one boot unit that includes one or more resilient materials,wherein at least some of the plurality of disk-drive connectors are eachin contact with a boot unit. In some embodiments, the apparatus includesat least one boot unit having graded shock absorbance characteristics.In some embodiments, the apparatus includes at least one boot unit thatincludes a vibration damping polymer. In some embodiments of theapparatus, the over-shock detector is further operable to store timeinformation regarding the over-shock events.

The invention provides a method that includes analyzingvibration-interaction between a plurality of disk drives held in anenclosure, and storing information that is based on the analysis into adata structure. In some embodiments, the method includes reading thestored information and adjusting a timing of at least one seek operationbased on the information.

The invention provides a method that includes mounting a plurality ofdisk drives to disk-drive connectors within an enclosure, adhering aresilient sheet across the plurality of disk drives, and attaching acover to the resilient sheet. In some embodiments of the method,attaching of the cover further includes adhering the cover to theresilient sheet. In some embodiments, the resilient sheet is attached tothe cover before the resilient sheet is adhered to the plurality of diskdrives. In some embodiments, the method includes connecting each of theplurality of disk drives to a boot unit. In some embodiments, the methodincludes adjusting a height of the boot unit based on a vibrationcharacteristic of the plurality of disk drives. In some embodiments, themethod includes connecting each of the plurality of disk drives to itsown respective boot unit. In some embodiments, the method includesconnecting each of the plurality of disk drives to a plurality of bootunits. In some embodiments, the method includes connecting each of theplurality of disk drives to a vibration-absorbing member. In someembodiments, the method includes adjusting a height of thevibration-absorbing member based on a vibration characteristic of theplurality of disk drives. In some embodiments, the method includesconnecting each of the plurality of disk drives to its own respectivevibration-absorbing member. In some embodiments, the method includesconnecting each of the plurality of disk drives to a plurality ofvibration-absorbing members.

The invention provides an apparatus that includes a plurality of diskdrives mounted to disk-drive connectors within an enclosure, a resilientsheet across the plurality of disk drives, and a cover. In someembodiments, the cover is adhered to the resilient sheet. In someembodiments, the resilient sheet is attached to the cover before theresilient sheet is adhered to the plurality of disk drives. In someembodiments, each of the plurality of disk drives is connected to a bootunit. In some embodiments, a height of the boot unit is adjusted basedon a vibration characteristic of the plurality of disk drives. In someembodiments, each of the plurality of disk drives is connected to itsown respective boot unit. In some embodiments, each of the plurality ofdisk drives is connected to a plurality of boot units. In someembodiments, each of the plurality of disk drives is connected to avibration-absorbing member. In some embodiments, a height of thevibration-absorbing member is adjusted based on a vibrationcharacteristic of the plurality of disk drives. In some embodiments,each of the plurality of disk drives is connected to its own respectivevibration-absorbing member. In some embodiments, each of the pluralityof disk drives is connected to a plurality of vibration-absorbingmembers.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Although numerous characteristics andadvantages of various embodiments as described herein have been setforth in the foregoing description, together with details of thestructure and function of various embodiments, many other embodimentsand changes to details will be apparent to those of skill in the artupon reviewing the above description. The scope of the invention shouldbe, therefore, determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled. Inthe appended claims, the terms “including” and “in which” are used asthe plain-English equivalents of the respective terms “comprising” and“wherein,” respectively. Moreover, the terms “first,” “second,” and“third,” etc., are used merely as labels, and are not intended to imposenumerical requirements on their objects.

1. A method comprising: providing a single drive enclosure holding aplurality of disk drives including a first, a second, a third, and afourth drive in a first row of disk drives, wherein the first rowextends generally from an air-inlet side of the enclosure to anair-outlet side of the enclosure, wherein the first and second driveshave major faces parallel to and at least as close to one another asthey are to any other drives, the second and third drives have majorfaces parallel to and at least as close to one another as they are toany other drives, and the third and fourth drives have major facesparallel to and at least as close to one another as they are to anyother drives, and wherein an air-inlet manifold extends along a firstside of the first row and an air-outlet manifold extends along anopposite second side of the first row; flowing air generally from anair-inlet side of the enclosure into and at least partially through theair-inlet manifold in a direction generally toward an air-outlet side ofthe enclosure along a first side of the first row; flowing air betweenthe first and second drives from the air-inlet manifold along the firstside of the first row to the air-outlet manifold along a second side ofthe first row, flowing air between the second and third drives from theair-inlet manifold along the first side of the first row to theair-outlet manifold along the second side of the first row, and flowingair between the third and fourth drives from the air-inlet manifoldalong the first side of the first row to the air-outlet manifold alongthe second side of the first row.
 2. The method of claim 1, furthercomprising: providing a fifth, a sixth, a seventh, and an eighth drivein a second row of disk drives in the enclosure, the second rowextending generally from the air-inlet side of the enclosure to theair-outlet side of the enclosure, wherein the fifth and sixth driveshave major faces parallel to and at least as close to one another asthey are to any other drives, the sixth and seventh drives have majorfaces parallel to and at least as close to one another as they are toany other drives, and the seventh and eighth drives have major facesparallel to and at least as close to one another as they are to anyother drives, and wherein the air-inlet manifold that extends along thefirst side of the first row also extends along a first side of thesecond row, and an air-outlet manifold extends along an opposite secondside of the second row; and flowing air between the fifth and sixthdrives from the air-inlet manifold along the first side of the secondrow to the air-outlet manifold along a second side of the second row,flowing air between the sixth and seventh drives from the air-inletmanifold along the first side of the second row to the air-outletmanifold along a second side of the second row, and flowing air betweenthe seventh and eighth drives from the first side of the second row tothe second side of the second row.
 3. The method of claim 2, wherein thefirst drive is at a non-parallel angle to the fifth drive, the seconddrive is at a non-parallel angle to the sixth drive, the third drive isat a non-parallel angle to the seventh drive, and the fourth drive is ata non-parallel angle to the eighth drive.
 4. The method of claim 2,further comprising: orienting the fifth and sixth drives such that theactuators are in a counter-rotating orientation to one another; andorienting the seventh and eighth drives such that the actuators are in acounter-rotating orientation to one another.
 5. The method of claim 4,further comprising: orienting the first row non-parallel to the secondrow.
 6. The method of claim 1, further comprising: rotating disks of thefirst disk drive in a first rotational direction; and rotating disks ofthe second disk drive in a counter-rotating direction to the firstrotational direction of the disks of the first disk drive.
 7. Anapparatus comprising: a drive enclosure configured to hold a pluralityof disk drives, the enclosure having a first plurality of disk-driveconnectors attached to a substrate; wherein the first plurality ofdisk-drive connectors include a first, a second, a third, and a fourthdisk-drive connectors in a first row, wherein the first row extendsgenerally from an air-inlet side of the enclosure to an air-outlet sideof the enclosure, wherein the first and second disk-drive connectorshave major faces parallel to and at least as close to one another asthey are to any other disk-drive connectors, the second and thirddisk-drive connectors have major faces parallel to and at least as closeto one another as they are to any other disk-drive connectors, and thethird and fourth disk-drive connectors have major faces parallel to andat least as close to one another as they are to any other disk-driveconnectors, and wherein an air-inlet manifold extends along a first sideof the first row and an air-outlet manifold extends along an oppositesecond side of the first row; wherein each one of the first plurality ofdisk-drive connectors connects to at least one disk drive; wherein theenclosure is configured to flow air generally from an air-inlet side ofthe enclosure into and at least partially through the air-inlet manifoldin a direction generally toward an air-outlet side of the enclosurealong a first side of the first row, and to flow air between the firstand second drives of each one of the first plurality of disk-driveconnectors from the air-inlet manifold along the first side of the firstrow to the air-outlet manifold along a second side of the first row. 8.The apparatus of claim 7, further comprising: a second plurality ofdisk-drive connectors attached to a substrate, wherein each one of thesecond plurality of disk-drive connectors is configured to receive atleast one disk drive; wherein the second plurality of disk-driveconnectors include a fifth, a sixth, a seventh, and an eighth disk-driveconnectors in a second row, wherein the second row extends generallyfrom an air-inlet side of the enclosure to an air-outlet side of theenclosure; wherein the fifth and sixth disk-drive connectors have majorfaces parallel to and at least as close to one another as they are toany other disk-drive connectors, the sixth and seventh disk-driveconnectors have major faces parallel to and at least as close to oneanother as they are to any other disk-drive connectors, and the seventhand eighth disk-drive connectors have major faces parallel to and atleast as close to one another as they are to any other disk-driveconnectors; wherein an air-inlet manifold extends along a first side ofthe second row and an air-outlet manifold extends along an oppositesecond side of the second row; and wherein the enclosure is configuredto flow air generally from the air-inlet side of the enclosure into andat least partially through the air-inlet manifold along the first sideof the second row, then between disk drives, when the disk drives areconnected to the second plurality of disk-drive connectors, to theair-outlet manifold along the second side of the second row.
 9. Theapparatus of claim 8, wherein: the first disk-drive connector is at anon-parallel angle to the fifth disk-drive connector, the seconddisk-drive connector is at a non-parallel angle to the sixth disk-driveconnector, the third disk-drive connector is at a non-parallel angle tothe seventh disk-drive connector, and the fourth disk-drive connector isat a non-parallel angle to the eighth disk-drive connector.
 10. Theapparatus of claim 8, wherein the disk-drive connectors for fifth andsixth drives are substantially parallel to one another and oriented suchthat the actuators are driven in counter-rotating directions to oneanother.
 11. The apparatus of claim 8, wherein the first row is orientednon-parallel to the second row.
 12. The apparatus of claim 8, whereinthe disk-drive connectors in the first row are oriented non-parallel tothe disk-drive connectors in the second row.
 13. The apparatus of claim7, wherein: a rotation of disks of the first disk drive connected tofirst disk-drive connector is in a first rotational direction; and arotation of disks of the second disk drive connected to seconddisk-drive connector is in a counter-rotating direction to the rotationof the disks of the first disk drive.
 14. The apparatus of claim 7,wherein the disk-drive connectors are oriented as a plurality of pairsof disk-drive connectors, wherein the disk-drive connectors within eachpair are laterally offset from one another.
 15. An apparatus comprising:a single drive enclosure holding a plurality of disk drives including afirst, a second, a third, and a fourth drive in a first row of diskdrives, wherein the first row extends generally from an air-inlet sideof the enclosure to an air-outlet side of the enclosure, wherein thefirst and second drives have major faces parallel to and at least asclose to one another as they are to any other drives, the second andthird drives have major faces parallel to and at least as close to oneanother as they are to any other drives, and the third and fourth driveshave major faces parallel to and at least as close to one another asthey are to any other drives, and wherein an air-inlet manifold extendsalong a first side of the first row and an air-outlet manifold extendsalong an opposite second side of the first row; means for flowing airgenerally from an air-inlet side of the enclosure into and at leastpartially through the air-inlet manifold in a direction generally towardan air-outlet side of the enclosure along a first side of the first row;means for flowing air between the first and second drives from theair-inlet manifold along the first side of the first row to theair-outlet manifold along a second side of the first row, means forflowing air between the second and third drives from the air-inletmanifold along the first side of the first row to the air-outletmanifold along the second side of the first row, and means for flowingair between the third and fourth drives from the air-inlet manifoldalong the first side of the first row to the air-outlet manifold alongthe second side of the first row.
 16. The apparatus of claim 15, furthercomprising: a fifth, a sixth, a seventh, and an eighth drive in a secondrow of disk drives in the enclosure, the second row extending generallyfrom the air-inlet side of the enclosure to the air-outlet side of theenclosure, wherein the fifth and sixth drives have major faces parallelto and at least as close to one another as they are to any other drives,the sixth and seventh drives have major faces parallel to and at leastas close to one another as they are to any other drives, and the seventhand eighth drives have major faces parallel to and at least as close toone another as they are to any other drives, and wherein the air-inletmanifold that extends along the first side of the first row also extendsalong a first side of the second row, and an air-outlet manifold extendsalong an opposite second side of the second row; and means for flowingair between the fifth and sixth drives from the air-inlet manifold alongthe first side of the second row to the air-outlet manifold along asecond side of the second row, means for flowing air between the sixthand seventh drives from the air-inlet manifold along the first side ofthe second row to the air-outlet manifold along a second side of thesecond row, and means for flowing air between the seventh and eighthdrives from the first side of the second row to the second side of thesecond row.
 17. The apparatus of claim 16, wherein the first drive is ata non-parallel angle to the fifth drive, the second drive is at anon-parallel angle to the sixth drive, the third drive is at anon-parallel angle to the seventh drive, and the fourth drive is at anon-parallel angle to the eighth drive.
 18. The apparatus of claim 16,wherein: the fifth and sixth drives are oriented such that the actuatorsare in a counter-rotating orientation to one another; and the seventhand eighth drives are oriented such that the actuators are in acounter-rotating orientation to one another.
 19. The apparatus of claim15, wherein the disk drives are oriented as a plurality of pairs of diskdrives, wherein the disk drives within each pair are laterally offsetfrom one another.
 20. The method of claim 1, further comprising:orienting the disk drives as a plurality of pairs of disk drives,wherein the disk drives within each pair are laterally offset from oneanother.